Jul 23 03:53:36.343911 ip-10-0-131-0 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 03:53:36.343920 ip-10-0-131-0 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 03:53:36.343927 ip-10-0-131-0 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 03:53:36.344145 ip-10-0-131-0 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 03:53:46.529943 ip-10-0-131-0 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 03:53:46.529962 ip-10-0-131-0 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 04df1b65d1da4ae1bf54504992ee9143 --
Jul 23 03:55:39.183685 ip-10-0-131-0 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 03:55:39.516956 ip-10-0-131-0 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 03:55:39.516956 ip-10-0-131-0 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 03:55:39.516956 ip-10-0-131-0 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 03:55:39.516956 ip-10-0-131-0 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 03:55:39.516956 ip-10-0-131-0 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 03:55:39.517773 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.517639    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 03:55:39.521083 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521068    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:39.521083 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521084    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521088    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521092    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521095    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521098    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521101    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521105    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521108    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521110    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521113    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521118    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521122    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521125    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521128    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521131    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521134    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521136    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521139    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521141    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:39.521152 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521144    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521147    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521150    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521152    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521155    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521158    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521160    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521163    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521166    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521168    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521171    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521173    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521176    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521179    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521181    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521184    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521187    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521189    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521192    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:39.521631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521195    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521197    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521200    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521202    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521204    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521207    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521209    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521212    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521214    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521217    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521220    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521223    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521226    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521228    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521230    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521234    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521236    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521239    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521241    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521244    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:39.522088 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521246    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521248    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521251    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521253    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521256    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521258    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521261    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521263    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521265    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521268    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521271    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521273    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521276    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521279    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521282    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521284    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521287    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521289    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521292    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521296    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:39.522631 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521299    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521301    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521304    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521306    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521309    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521312    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521314    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521318    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521320    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521323    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521325    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521328    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521330    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521333    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521335    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521337    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521340    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521342    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521345    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.521347    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:39.523111 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522043    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522049    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522052    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522055    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522058    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522061    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522063    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522066    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522068    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522071    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522074    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522077    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522079    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522082    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522084    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522087    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522089    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522092    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522095    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522098    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:39.523601 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522101    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522105    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522109    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522111    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522114    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522117    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522120    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522122    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522126    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522129    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522132    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522134    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522137    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522139    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522141    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522144    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522146    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522149    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522152    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:39.524109 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522154    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522157    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522159    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522163    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522165    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522168    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522171    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522173    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522176    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522178    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522180    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522183    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522185    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522188    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522190    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522192    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522195    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522197    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522200    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:39.524577 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522202    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522205    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522207    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522209    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522212    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522214    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522217    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522220    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522222    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522224    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522227    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522229    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522232    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522235    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522238    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522241    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522243    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522246    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522248    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522251    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:39.525045 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522253    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522255    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522258    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522260    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522262    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522265    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522267    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522269    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522272    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522275    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522277    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522279    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522282    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522284    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522286    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522291    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522294    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522297    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522299    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522302    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:39.525553 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522305    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522695    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522701    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522704    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522707    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522710    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522718    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522722    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522726    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522729    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522732    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522735    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522738    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522740    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522743    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522746    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522749    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522751    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522753    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:39.526061 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522756    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522758    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522761    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522763    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522766    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522768    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522771    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522773    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522776    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522778    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522781    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522783    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522785    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522788    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522790    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522793    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522795    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522797    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522803    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522807    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:39.526636 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522809    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522814    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522817    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522820    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522822    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522825    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522827    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522830    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522832    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522834    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522837    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522839    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522841    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522844    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522847    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522849    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522851    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522854    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522856    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522859    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:39.527130 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522861    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522863    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522866    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522868    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522871    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522873    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522875    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522877    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522880    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522882    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522886    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522890    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522893    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522896    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522899    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522901    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522904    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522906    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522909    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:39.527664 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522911    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522914    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522916    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522918    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522921    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522923    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522926    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522928    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522930    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522934    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522936    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522939    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522941    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522944    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522946    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522948    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522951    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522953    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522955    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522958    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:39.528135 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522960    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.522962    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523026    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523035    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523042    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523046    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523050    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523054    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523058    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523062    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523065    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523068    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523071    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523074    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523077    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523080    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523083    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523086    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523089    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523091    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523095    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523098    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523101    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523104    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 03:55:39.528635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523107    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523110    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523113    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523116    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523119    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523122    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523125    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523128    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523131    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523134    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523137    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523141    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523146    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523148    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523151    2563 flags.go:64] FLAG: --enable-server="true"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523155    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523159    2563 flags.go:64] FLAG: --event-burst="100"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523162    2563 flags.go:64] FLAG: --event-qps="50"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523165    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523168    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523171    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523174    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523177    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523180    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523183    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 23 03:55:39.529205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523186    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523188    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523191    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523194    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523196    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523199    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523202    2563 flags.go:64] FLAG: --feature-gates=""
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523205    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523208    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523210    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523213    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523216    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523219    2563 flags.go:64] FLAG: --help="false"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523221    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523224    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523227    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523230    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523233    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523237    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523241    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523243    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523246    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523249    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 03:55:39.529816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523253    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523255    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523258    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523261    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523263    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523266    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523269    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523271    2563 flags.go:64] FLAG: --lock-file=""
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523274    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523277    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523279    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523284    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523287    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523290    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523292    2563 flags.go:64] FLAG: --logging-format="text"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523295    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523298    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523300    2563 flags.go:64] FLAG: --manifest-url=""
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523303    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523308    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523310    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523317    2563 flags.go:64] FLAG: --max-pods="110"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523320    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523323    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523325    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 03:55:39.530399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523328    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523331    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523335    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523338    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523345    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523348    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523351    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523353    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523357    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523362    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523365    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523368    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523371    2563 flags.go:64] FLAG: --port="10250"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523373    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523376    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b1b9ec68b86e0687"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523379    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523382    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523385    2563 flags.go:64] FLAG: --register-node="true"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523388    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523391    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523394    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523396    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523399    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523402    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523419    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 03:55:39.531058 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523423    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523426    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523428    2563 flags.go:64] FLAG: --runonce="false"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523431    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523434    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523437    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523440    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523442    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523445    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523448    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523453    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523456    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523458    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523461    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523464    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523467    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523470    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523473    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523478    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523481    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523483    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523487    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523490    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523492    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523495    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 03:55:39.531684 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523498    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523500    2563 flags.go:64] FLAG: --v="2"
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523504    2563 flags.go:64] FLAG: --version="false"
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523508    2563 flags.go:64] FLAG: --vmodule=""
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523511    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.523514    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523621    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523625    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523628    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523631    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523634    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523637    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523640    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523642    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523644    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523647    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523650    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523655    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523658    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523661    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523664    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:39.532297 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523666    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523669    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523672    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523676    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523679    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523681    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523683    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523687    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523690    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523693    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523696    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523699    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523701    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523704    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523706    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523709    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523712    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523714    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523717    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:39.532810 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523720    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523722    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523725    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523728    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523730    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523732    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523735    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523737    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523740    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523743    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523746    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523748    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523751    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523753    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523756    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523758    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523761    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523764    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523766    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523769    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:39.533264 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523771    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523774    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523776    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523779    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523782    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523784    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523787    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523789    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523791    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523794    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523796    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523799    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523801    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523804    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523806    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523809    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523811    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523814    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523816    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523818    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:39.533755 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523821    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523824    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523826    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523828    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523831    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523833    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523835    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523838    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523840    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523842    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523845    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523847    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523850    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523860    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523864    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523867    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523869    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523872    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523875    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:39.534240 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523877    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523879    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523882    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523884    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523887    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.523889    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.524428    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.531089    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.531104    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.531791    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.531799    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531871    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531876    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531880    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:39.534712 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531883    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531886    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531888    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531891    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531894    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531897    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531899    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531902    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531904    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531913    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531916    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531920    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531924    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531927    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531929    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531932    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531935    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531938    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531940    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531942    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:39.535068 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531945    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531947    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531950    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531952    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531955    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531958    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531961    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531963    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531966    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531969    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531971    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531973    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531976    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531978    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531981    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531983    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531986    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531988    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531990    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531993    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:39.535578 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531995    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.531997    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532005    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532008    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532012    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532015    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532018    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532021    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532023    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532026    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532028    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532031    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532033    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532035    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532038    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532040    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532043    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532045    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532049    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:39.536081 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532051    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532054    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532056    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532059    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532061    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532064    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532066    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532069    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532071    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532073    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532075    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532078    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532080    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532082    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532085    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532087    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532097    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532100    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532102    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532104    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:39.536570 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532108    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532111    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532114    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532117    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532120    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532122    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532124    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532127    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532129    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532131    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532134    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532137    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532139    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532142    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532145    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532147    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:39.537054 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532149    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.532154    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532314    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532319    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532322    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532325    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532328    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532330    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532333    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532335    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532337    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532340    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532343    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532350    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532353    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:39.537458 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532356    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532358    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532360    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532363    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532365    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532368    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532372    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532375    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532378    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532380    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532382    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532385    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532387    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532390    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532393    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532395    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532397    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532400    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532402    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:39.537820 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532405    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532424    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532427    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532429    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532432    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532434    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532437    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532439    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532441    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532444    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532446    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532448    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532457    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532460    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532462    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532464    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532468    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532471    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532474    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:39.538277 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532477    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532479    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532482    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532485    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532487    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532490    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532493    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532496    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532498    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532501    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532503    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532505    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532508    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532510    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532513    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532515    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532518    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532520    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532523    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532526    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:39.538787 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532530    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532532    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532535    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532537    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532539    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532542    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532550    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532552    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532555    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532557    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532559    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532562    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532564    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532567    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532569    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532571    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532573    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532576    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532578    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532581    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532583    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:39.539273 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532586    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532588    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532590    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532593    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532595    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532597    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:39.532600    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.532604    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.532706    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.536291    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.537444    2563 server.go:1019] "Starting client certificate rotation"
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.537540    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 03:55:39.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.537594    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 03:55:39.541590 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.541569    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 03:55:39.545881 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.545862    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 03:55:39.556766 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.556721    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 23 03:55:39.558815 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.558802    2563 log.go:25] "Validated CRI v1 image API"
Jul 23 03:55:39.559551 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.559537    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 03:55:39.561730 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.561708    2563 fs.go:135] Filesystem UUIDs: map[0494248b-a951-4f34-b259-b1960b2f1fb2:/dev/nvme0n1p4 0ec309b5-3ed9-4b79-9196-063187b091a1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 23 03:55:39.561805 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.561730    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 03:55:39.567082 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.566973    2563 manager.go:216] Machine: {Timestamp:2026-07-23 03:55:39.565749252 +0000 UTC m=+0.285280854 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102911 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c138c156764d5ad92ca0fc2319001 SystemUUID:ec2c138c-1567-64d5-ad92-ca0fc2319001 BootID:04df1b65-d1da-4ae1-bf54-504992ee9143 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0e:ea:9e:bf:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0e:ea:9e:bf:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:3f:ca:d9:6e:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 03:55:39.567082 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.567070    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 03:55:39.567237 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.567172    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 03:55:39.567287 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.567256    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 03:55:39.569813 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.569781    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 03:55:39.569972 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.569816    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-0.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 03:55:39.570546 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.570534    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 03:55:39.570610 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.570551    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 03:55:39.570610 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.570561    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 03:55:39.571139 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.571128    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 03:55:39.572715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.572703    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 03:55:39.576590 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576575    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 03:55:39.576729 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576718    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 03:55:39.576792 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576766    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 03:55:39.576847 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576798    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 03:55:39.576847 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576819    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 03:55:39.576847 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576832    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 23 03:55:39.576847 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.576844    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 03:55:39.577246 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.577232    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 03:55:39.577316 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.577269    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 03:55:39.579564 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.579546    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 03:55:39.582894 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.582870    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 03:55:39.582987 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.582922    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 03:55:39.584577 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584561    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584581    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584587    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584594    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584606    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584615    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584626    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584631    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584638    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584644    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584653    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 03:55:39.584660 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.584658    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 03:55:39.585310 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.585293    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 03:55:39.585310 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.585306    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 03:55:39.586510 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.586488    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 03:55:39.586581 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.586508    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 03:55:39.588514 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.588501    2563 server.go:1268] "Started kubelet"
Jul 23 03:55:39.588622 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.588588    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 03:55:39.588709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.588589    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 03:55:39.588709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.588660    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 03:55:39.589306 ip-10-0-131-0 systemd[1]: Started Kubernetes Kubelet.
Jul 23 03:55:39.590341 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.590325    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 03:55:39.591942 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.591924    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2hlhq"
Jul 23 03:55:39.593053 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.593038    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 03:55:39.598826 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.598808    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 03:55:39.598908 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.598826    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 03:55:39.598975 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.598938    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:39.599466 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.599450    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 03:55:39.599466 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.599452    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 03:55:39.599565 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.599473    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 03:55:39.599565 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.599554    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 03:55:39.599565 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.599559    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 03:55:39.600422 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.600390    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2hlhq"
Jul 23 03:55:39.600625 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.600605    2563 factory.go:55] Registering systemd factory
Jul 23 03:55:39.600625 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.600626    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 23 03:55:39.600981 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.600958    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 03:55:39.601080 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.601069    2563 factory.go:153] Registering CRI-O factory
Jul 23 03:55:39.601134 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.601082    2563 factory.go:223] Registration of the crio container factory successfully
Jul 23 03:55:39.601134 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.601129    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 03:55:39.601207 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.601147    2563 factory.go:103] Registering Raw factory
Jul 23 03:55:39.601207 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.601156    2563 manager.go:1196] Started watching for new ooms in manager
Jul 23 03:55:39.602287 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.602248    2563 manager.go:349] Starting recovery of all containers
Jul 23 03:55:39.606537 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.606516    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-0.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 03:55:39.606724 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.606702    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 03:55:39.607541 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.607517    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 03:55:39.607606 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.606474    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-0.ec2.internal.18c4cefd48c4eb69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-0.ec2.internal,UID:ip-10-0-131-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-0.ec2.internal,},FirstTimestamp:2026-07-23 03:55:39.588471657 +0000 UTC m=+0.308003258,LastTimestamp:2026-07-23 03:55:39.588471657 +0000 UTC m=+0.308003258,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-0.ec2.internal,}"
Jul 23 03:55:39.614558 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.614454    2563 manager.go:354] Recovery completed
Jul 23 03:55:39.618442 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.618429    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.620098 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620081    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.620153 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620111    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.620153 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620124    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.620649 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620636    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 03:55:39.620649 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620648    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 03:55:39.620736 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620662    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 03:55:39.620736 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.620701    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.621434 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.621403    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.621483 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.621446    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.621483 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.621456    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.621961 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.621904    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-0.ec2.internal.18c4cefd4aa77cab  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-0.ec2.internal,UID:ip-10-0-131-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-0.ec2.internal,},FirstTimestamp:2026-07-23 03:55:39.620097195 +0000 UTC m=+0.339628797,LastTimestamp:2026-07-23 03:55:39.620097195 +0000 UTC m=+0.339628797,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-0.ec2.internal,}"
Jul 23 03:55:39.622769 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.622734    2563 policy_none.go:49] "None policy: Start"
Jul 23 03:55:39.622769 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.622751    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 03:55:39.622769 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.622760    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 03:55:39.624769 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.624757    2563 policy_none.go:47] "Start"
Jul 23 03:55:39.659506 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659492    2563 manager.go:337] "Starting Device Plugin manager"
Jul 23 03:55:39.659558 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.659533    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 03:55:39.659558 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659546    2563 server.go:87] "Starting device plugin registration server"
Jul 23 03:55:39.659789 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659778    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 03:55:39.659833 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659791    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 03:55:39.659907 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659885    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 03:55:39.659996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659970    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 03:55:39.659996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.659984    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 03:55:39.660454 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.660433    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 03:55:39.660524 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.660487    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:39.742602 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.742575    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 03:55:39.744759 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.743865    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 03:55:39.744759 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.743900    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 03:55:39.744759 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.743927    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 03:55:39.744759 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.743962    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 03:55:39.746379 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.746358    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:39.760391 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.760360    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.761119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.761105    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.761169 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.761136    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.761169 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.761151    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.761250 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.761173    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.767358 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.767307    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.767358 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.767331    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-0.ec2.internal\": node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:39.781446 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.781402    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:39.844471 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.844447    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"]
Jul 23 03:55:39.844568 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.844521    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.845824 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.845809    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.845899 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.845840    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.845899 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.845854    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.848107 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848091    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.848213 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.848257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848231    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.848777 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848762    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.848823 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848792    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.848823 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848801    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.848883 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848764    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.848883 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848864    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.848883 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.848873    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.851053 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.851037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.851139 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.851069    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:39.851720 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.851704    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:39.851800 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.851735    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:39.851800 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:39.851750    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:39.874967 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.874948    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-0.ec2.internal\" not found" node="ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.879147 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.879132    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-0.ec2.internal\" not found" node="ip-10-0-131-0.ec2.internal"
Jul 23 03:55:39.882189 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.882174    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:39.982950 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:39.982916    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:40.001226 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.001203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb1187c486ec4bc41761dae0bcf5f62e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal\" (UID: \"bb1187c486ec4bc41761dae0bcf5f62e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.001304 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.001246    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb1187c486ec4bc41761dae0bcf5f62e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal\" (UID: \"bb1187c486ec4bc41761dae0bcf5f62e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.001304 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.001268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df25c8e9f4a25a3267946e74f3761d4c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-0.ec2.internal\" (UID: \"df25c8e9f4a25a3267946e74f3761d4c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.083690 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.083614    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:40.101964 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.101939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb1187c486ec4bc41761dae0bcf5f62e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal\" (UID: \"bb1187c486ec4bc41761dae0bcf5f62e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.102062 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.101992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb1187c486ec4bc41761dae0bcf5f62e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal\" (UID: \"bb1187c486ec4bc41761dae0bcf5f62e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.102062 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.102035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df25c8e9f4a25a3267946e74f3761d4c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-0.ec2.internal\" (UID: \"df25c8e9f4a25a3267946e74f3761d4c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.102158 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.102042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb1187c486ec4bc41761dae0bcf5f62e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal\" (UID: \"bb1187c486ec4bc41761dae0bcf5f62e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.102158 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.102099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df25c8e9f4a25a3267946e74f3761d4c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-0.ec2.internal\" (UID: \"df25c8e9f4a25a3267946e74f3761d4c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.102158 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.102048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb1187c486ec4bc41761dae0bcf5f62e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal\" (UID: \"bb1187c486ec4bc41761dae0bcf5f62e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.178152 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.178114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.181939 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.181920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.184630 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.184610    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:40.284782 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.284729    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:40.385296 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.385225    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-0.ec2.internal\" not found"
Jul 23 03:55:40.432998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.432967    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:40.499003 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.498977    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.509089 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.509069    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 03:55:40.510519 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.510506    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal"
Jul 23 03:55:40.519959 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.519944    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 03:55:40.537088 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.537073    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 03:55:40.537198 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.537179    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 03:55:40.537266 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.537225    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 03:55:40.577370 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.577347    2563 apiserver.go:52] "Watching apiserver"
Jul 23 03:55:40.594363 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.594344    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 03:55:40.594785 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.594760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6","openshift-dns/node-resolver-zqk29","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal","openshift-multus/multus-additional-cni-plugins-78qc6","openshift-multus/multus-zsl88","openshift-multus/network-metrics-daemon-6zfrb","kube-system/global-pull-secret-syncer-z9cwq","openshift-cluster-node-tuning-operator/tuned-798dd","openshift-image-registry/node-ca-p47ct","openshift-network-diagnostics/network-check-target-q9k5f","openshift-network-operator/iptables-alerter-8qdjx","openshift-ovn-kubernetes/ovnkube-node-9qstf","kube-system/konnectivity-agent-rcdxk","kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal"]
Jul 23 03:55:40.597857 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.597841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.597936 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.597918    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:40.599550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.599536    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 03:55:40.599816 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.599797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.602117 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.602096    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 03:55:40.602214 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.602107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5vkwf\""
Jul 23 03:55:40.602214 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.602138    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 03:55:40.602214 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.602178    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.604170 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604156    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.604248 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604232    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ds9tf\""
Jul 23 03:55:40.604674 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4052be7b-0b50-40e8-a250-3e251a80d7d9-dbus\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.604765 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604666    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.604765 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.604765 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3e165c3-a86b-4cd8-808f-67726369adb2-agent-certs\") pod \"konnectivity-agent-rcdxk\" (UID: \"b3e165c3-a86b-4cd8-808f-67726369adb2\") " pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.604765 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3e165c3-a86b-4cd8-808f-67726369adb2-konnectivity-ca\") pod \"konnectivity-agent-rcdxk\" (UID: \"b3e165c3-a86b-4cd8-808f-67726369adb2\") " pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.604986 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4052be7b-0b50-40e8-a250-3e251a80d7d9-kubelet-config\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.604986 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.604955    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.606979 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.606958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.608756 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.608740    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 03:55:40.608962 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.608928    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 03:50:39 +0000 UTC" deadline="2028-01-11 20:57:44.904368345 +0000 UTC"
Jul 23 03:55:40.608962 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.608925    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 03:55:40.609094 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.608985    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.609094 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.609025    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.609094 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.608963    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12905h2m4.295408318s"
Jul 23 03:55:40.609240 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.609097    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nvwm8\""
Jul 23 03:55:40.609240 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.609177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:40.609337 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.609236    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:40.609337 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.609296    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 03:55:40.609880 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.609853    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zgj4r\""
Jul 23 03:55:40.610159 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.610145    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 03:55:40.611788 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.611775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.612285 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.612270    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 03:55:40.613911 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.613892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 03:55:40.614009 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.613954    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q7pb9\""
Jul 23 03:55:40.614009 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.613975    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.614009 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.613979    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.614490 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.614475    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.616647 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.616630    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gz7wj\""
Jul 23 03:55:40.616729 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.616697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 03:55:40.616948 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.616932    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.617130 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.617108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.617274 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.617260    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.619033 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.619010    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.619200 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.619184    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.619269 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.619259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hwrkm\""
Jul 23 03:55:40.619781 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.619765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:40.619874 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.619851    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:40.622442 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.622425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.624599 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.624584    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.627669 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.625322    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 03:55:40.627669 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.625360    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.627669 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.625428    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t4w9j\""
Jul 23 03:55:40.627669 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.626127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.629483 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.629459    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 03:55:40.629583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.629558    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 03:55:40.629583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.629573    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 03:55:40.629714 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.629699    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 03:55:40.629812 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.629797    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 03:55:40.629927 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.629817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j4vp2\""
Jul 23 03:55:40.630310 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.630298    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 03:55:40.655472 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.655455    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-csx9k"
Jul 23 03:55:40.663604 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.663579    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-csx9k"
Jul 23 03:55:40.701162 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.701141    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 03:55:40.705292 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-cni-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.705376 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-cni-multus\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.705376 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w95bc\" (UniqueName: \"kubernetes.io/projected/17763bad-e4bc-46cb-bd38-dcbb27240c1f-kube-api-access-w95bc\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:40.705376 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-socket-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.705376 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.705534 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.705534 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.705501    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:40.705613 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysconfig\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.705613 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.705578    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:41.205557193 +0000 UTC m=+1.925088801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:40.705613 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-run\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.705725 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-socket-dir-parent\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.705725 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-sys-fs\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.705725 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-ovn\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.705725 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4052be7b-0b50-40e8-a250-3e251a80d7d9-dbus\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.705725 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-cni-binary-copy\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wnx\" (UniqueName: \"kubernetes.io/projected/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-kube-api-access-q7wnx\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-device-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw25x\" (UniqueName: \"kubernetes.io/projected/aa7bbf7b-4941-4be6-9966-b794b4915ff7-kube-api-access-qw25x\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20894936-ef54-43da-8782-fe2fbdf46bb0-iptables-alerter-script\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzbl7\" (UniqueName: \"kubernetes.io/projected/20894936-ef54-43da-8782-fe2fbdf46bb0-kube-api-access-bzbl7\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4052be7b-0b50-40e8-a250-3e251a80d7d9-dbus\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysctl-conf\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-var-lib-kubelet\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17528e33-d6b3-4010-9888-08949fa8f635-tmp\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.705930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-kubelet\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-systemd\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.705986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-node-log\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6fq\" (UniqueName: \"kubernetes.io/projected/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-kube-api-access-vt6fq\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3e165c3-a86b-4cd8-808f-67726369adb2-agent-certs\") pod \"konnectivity-agent-rcdxk\" (UID: \"b3e165c3-a86b-4cd8-808f-67726369adb2\") " pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-systemd\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-daemon-config\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1bbe5046-a390-4657-968b-f2943a0213b9-serviceca\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-var-lib-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovnkube-script-lib\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-os-release\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-cni-bin\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6kkw\" (UniqueName: \"kubernetes.io/projected/31c5a4f1-0afb-4208-b1a8-849405b3eea8-kube-api-access-t6kkw\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n6g9\" (UniqueName: \"kubernetes.io/projected/cb7d2494-e059-4efb-b6a0-87c444f8b948-kube-api-access-4n6g9\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-lib-modules\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.706335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8mv\" (UniqueName: \"kubernetes.io/projected/17528e33-d6b3-4010-9888-08949fa8f635-kube-api-access-kt8mv\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-slash\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706321    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17528e33-d6b3-4010-9888-08949fa8f635-etc-tuned\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhbg9\" (UniqueName: \"kubernetes.io/projected/1bbe5046-a390-4657-968b-f2943a0213b9-kube-api-access-dhbg9\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-kubelet\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-kubernetes\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-os-release\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-cni-netd\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-modprobe-d\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-k8s-cni-cncf-io\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-system-cni-dir\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-run-netns\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-sys\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-conf-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1bbe5046-a390-4657-968b-f2943a0213b9-host\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.707179 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-registration-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cnibin\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.706990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-etc-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-system-cni-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovnkube-config\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3e165c3-a86b-4cd8-808f-67726369adb2-konnectivity-ca\") pod \"konnectivity-agent-rcdxk\" (UID: \"b3e165c3-a86b-4cd8-808f-67726369adb2\") " pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707281    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysctl-d\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-hostroot\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb7d2494-e059-4efb-b6a0-87c444f8b948-tmp-dir\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-log-socket\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-cni-bin\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.707930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707486    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20894936-ef54-43da-8782-fe2fbdf46bb0-host-slash\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4052be7b-0b50-40e8-a250-3e251a80d7d9-kubelet-config\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-host\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-multus-certs\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4052be7b-0b50-40e8-a250-3e251a80d7d9-kubelet-config\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb7d2494-e059-4efb-b6a0-87c444f8b948-hosts-file\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-systemd-units\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-cnibin\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-netns\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-etc-kubernetes\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3e165c3-a86b-4cd8-808f-67726369adb2-konnectivity-ca\") pod \"konnectivity-agent-rcdxk\" (UID: \"b3e165c3-a86b-4cd8-808f-67726369adb2\") " pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cni-binary-copy\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.708477 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.707816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-env-overrides\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.709517 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.709496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3e165c3-a86b-4cd8-808f-67726369adb2-agent-certs\") pod \"konnectivity-agent-rcdxk\" (UID: \"b3e165c3-a86b-4cd8-808f-67726369adb2\") " pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.785986 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:40.785957    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf25c8e9f4a25a3267946e74f3761d4c.slice/crio-3de65e20fc0fdee051238f90f4fe1f8d8d6eca2cd004602111512f39d93e85cf WatchSource:0}: Error finding container 3de65e20fc0fdee051238f90f4fe1f8d8d6eca2cd004602111512f39d93e85cf: Status 404 returned error can't find the container with id 3de65e20fc0fdee051238f90f4fe1f8d8d6eca2cd004602111512f39d93e85cf
Jul 23 03:55:40.786174 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:40.786159    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1187c486ec4bc41761dae0bcf5f62e.slice/crio-d4998831a7fbbf93d95978887c52f861b8fa68ad400eecf8cecf9668a21bb7d6 WatchSource:0}: Error finding container d4998831a7fbbf93d95978887c52f861b8fa68ad400eecf8cecf9668a21bb7d6: Status 404 returned error can't find the container with id d4998831a7fbbf93d95978887c52f861b8fa68ad400eecf8cecf9668a21bb7d6
Jul 23 03:55:40.791453 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.791379    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 03:55:40.808176 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-host\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.808176 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-multus-certs\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb7d2494-e059-4efb-b6a0-87c444f8b948-hosts-file\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-systemd-units\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-cnibin\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-netns\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-host\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb7d2494-e059-4efb-b6a0-87c444f8b948-hosts-file\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808274    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-multus-certs\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-systemd-units\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-etc-kubernetes\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-netns\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-etc-kubernetes\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cni-binary-copy\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-cnibin\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-env-overrides\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-cni-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-cni-multus\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95bc\" (UniqueName: \"kubernetes.io/projected/17763bad-e4bc-46cb-bd38-dcbb27240c1f-kube-api-access-w95bc\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-socket-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808669    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-cni-multus\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysconfig\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-run\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-socket-dir-parent\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808782    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.808875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-sys-fs\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-socket-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysconfig\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-sys-fs\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-run\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-socket-dir-parent\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-ovn\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-ovn\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-cni-binary-copy\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wnx\" (UniqueName: \"kubernetes.io/projected/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-kube-api-access-q7wnx\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-cni-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.808987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-device-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw25x\" (UniqueName: \"kubernetes.io/projected/aa7bbf7b-4941-4be6-9966-b794b4915ff7-kube-api-access-qw25x\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20894936-ef54-43da-8782-fe2fbdf46bb0-iptables-alerter-script\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-env-overrides\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzbl7\" (UniqueName: \"kubernetes.io/projected/20894936-ef54-43da-8782-fe2fbdf46bb0-kube-api-access-bzbl7\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.809080    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:40.809709 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-device-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysctl-conf\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cni-binary-copy\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.809132    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:55:41.309116145 +0000 UTC m=+2.028647745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-var-lib-kubelet\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17528e33-d6b3-4010-9888-08949fa8f635-tmp\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-kubelet\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysctl-conf\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809261    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-kubelet\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-var-lib-kubelet\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-systemd\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-node-log\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809326    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-systemd\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6fq\" (UniqueName: \"kubernetes.io/projected/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-kube-api-access-vt6fq\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-node-log\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-systemd\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.810715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-daemon-config\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1bbe5046-a390-4657-968b-f2943a0213b9-serviceca\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-var-lib-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-cni-binary-copy\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovnkube-script-lib\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-var-lib-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-os-release\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-cni-bin\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-systemd\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kkw\" (UniqueName: \"kubernetes.io/projected/31c5a4f1-0afb-4208-b1a8-849405b3eea8-kube-api-access-t6kkw\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n6g9\" (UniqueName: \"kubernetes.io/projected/cb7d2494-e059-4efb-b6a0-87c444f8b948-kube-api-access-4n6g9\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-lib-modules\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8mv\" (UniqueName: \"kubernetes.io/projected/17528e33-d6b3-4010-9888-08949fa8f635-kube-api-access-kt8mv\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-slash\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17528e33-d6b3-4010-9888-08949fa8f635-etc-tuned\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhbg9\" (UniqueName: \"kubernetes.io/projected/1bbe5046-a390-4657-968b-f2943a0213b9-kube-api-access-dhbg9\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.811432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-kubelet\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-kubernetes\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-os-release\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-cni-netd\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-modprobe-d\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-k8s-cni-cncf-io\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-system-cni-dir\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-daemon-config\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-run-netns\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-sys\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-os-release\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.809991    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-kubelet\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-conf-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-var-lib-cni-bin\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-kubernetes\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-slash\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-cni-netd\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810118    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1bbe5046-a390-4657-968b-f2943a0213b9-serviceca\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.811996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovnkube-script-lib\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20894936-ef54-43da-8782-fe2fbdf46bb0-iptables-alerter-script\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-modprobe-d\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1bbe5046-a390-4657-968b-f2943a0213b9-host\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810191    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-lib-modules\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-host-run-k8s-cni-cncf-io\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1bbe5046-a390-4657-968b-f2943a0213b9-host\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810276    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-system-cni-dir\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-registration-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-sys\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-registration-dir\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-multus-conf-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-run-netns\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cnibin\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-os-release\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.812550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-etc-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31c5a4f1-0afb-4208-b1a8-849405b3eea8-cnibin\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-etc-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-system-cni-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-run-openvswitch\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-system-cni-dir\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovnkube-config\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysctl-d\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-hostroot\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa7bbf7b-4941-4be6-9966-b794b4915ff7-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb7d2494-e059-4efb-b6a0-87c444f8b948-tmp-dir\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-log-socket\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17528e33-d6b3-4010-9888-08949fa8f635-etc-sysctl-d\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-log-socket\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813083 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovnkube-config\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31c5a4f1-0afb-4208-b1a8-849405b3eea8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.810982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb7d2494-e059-4efb-b6a0-87c444f8b948-tmp-dir\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.811020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-hostroot\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.811102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-cni-bin\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.811226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-host-cni-bin\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.811254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.811270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20894936-ef54-43da-8782-fe2fbdf46bb0-host-slash\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.811332    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20894936-ef54-43da-8782-fe2fbdf46bb0-host-slash\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.812209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17528e33-d6b3-4010-9888-08949fa8f635-tmp\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.812308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17528e33-d6b3-4010-9888-08949fa8f635-etc-tuned\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.813618 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.813291    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.814845 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.814830    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:40.814910 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.814855    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:40.814910 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.814866    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:40.814974 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:40.814918    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:41.31490356 +0000 UTC m=+2.034435152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:40.817070 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.817043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95bc\" (UniqueName: \"kubernetes.io/projected/17763bad-e4bc-46cb-bd38-dcbb27240c1f-kube-api-access-w95bc\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:40.818484 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.818374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw25x\" (UniqueName: \"kubernetes.io/projected/aa7bbf7b-4941-4be6-9966-b794b4915ff7-kube-api-access-qw25x\") pod \"aws-ebs-csi-driver-node-nd2b6\" (UID: \"aa7bbf7b-4941-4be6-9966-b794b4915ff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:40.819868 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.819779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhbg9\" (UniqueName: \"kubernetes.io/projected/1bbe5046-a390-4657-968b-f2943a0213b9-kube-api-access-dhbg9\") pod \"node-ca-p47ct\" (UID: \"1bbe5046-a390-4657-968b-f2943a0213b9\") " pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:40.820137 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.820115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n6g9\" (UniqueName: \"kubernetes.io/projected/cb7d2494-e059-4efb-b6a0-87c444f8b948-kube-api-access-4n6g9\") pod \"node-resolver-zqk29\" (UID: \"cb7d2494-e059-4efb-b6a0-87c444f8b948\") " pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:40.820241 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.820228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6fq\" (UniqueName: \"kubernetes.io/projected/032d1254-cfe2-47d6-ba4d-80752c4f2ae4-kube-api-access-vt6fq\") pod \"ovnkube-node-9qstf\" (UID: \"032d1254-cfe2-47d6-ba4d-80752c4f2ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:40.820525 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.820505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzbl7\" (UniqueName: \"kubernetes.io/projected/20894936-ef54-43da-8782-fe2fbdf46bb0-kube-api-access-bzbl7\") pod \"iptables-alerter-8qdjx\" (UID: \"20894936-ef54-43da-8782-fe2fbdf46bb0\") " pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:40.820669 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.820635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6kkw\" (UniqueName: \"kubernetes.io/projected/31c5a4f1-0afb-4208-b1a8-849405b3eea8-kube-api-access-t6kkw\") pod \"multus-additional-cni-plugins-78qc6\" (UID: \"31c5a4f1-0afb-4208-b1a8-849405b3eea8\") " pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.820821 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.820805    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8mv\" (UniqueName: \"kubernetes.io/projected/17528e33-d6b3-4010-9888-08949fa8f635-kube-api-access-kt8mv\") pod \"tuned-798dd\" (UID: \"17528e33-d6b3-4010-9888-08949fa8f635\") " pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.820910 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.820894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wnx\" (UniqueName: \"kubernetes.io/projected/77281df7-2da2-4a68-a3dc-4b6b9f1d2455-kube-api-access-q7wnx\") pod \"multus-zsl88\" (UID: \"77281df7-2da2-4a68-a3dc-4b6b9f1d2455\") " pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.934435 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.934349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:55:40.944960 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:40.944932    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e165c3_a86b_4cd8_808f_67726369adb2.slice/crio-22e1930926cd500da4b8aaefd63428a72bed36008fd3d29deb486b8de2d0505b WatchSource:0}: Error finding container 22e1930926cd500da4b8aaefd63428a72bed36008fd3d29deb486b8de2d0505b: Status 404 returned error can't find the container with id 22e1930926cd500da4b8aaefd63428a72bed36008fd3d29deb486b8de2d0505b
Jul 23 03:55:40.948169 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.948129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-798dd"
Jul 23 03:55:40.952949 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.952929    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:40.956562 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:40.956540    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17528e33_d6b3_4010_9888_08949fa8f635.slice/crio-d37a06366b3c147c1edf290f45346437a952e98a0d48d9adbcd709a3a636bcfa WatchSource:0}: Error finding container d37a06366b3c147c1edf290f45346437a952e98a0d48d9adbcd709a3a636bcfa: Status 404 returned error can't find the container with id d37a06366b3c147c1edf290f45346437a952e98a0d48d9adbcd709a3a636bcfa
Jul 23 03:55:40.962565 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.962539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-78qc6"
Jul 23 03:55:40.970421 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:40.970383    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c5a4f1_0afb_4208_b1a8_849405b3eea8.slice/crio-19fac2d455ee9c52524521a4e3e529d3d957e191c169f967e5b5ebced197931b WatchSource:0}: Error finding container 19fac2d455ee9c52524521a4e3e529d3d957e191c169f967e5b5ebced197931b: Status 404 returned error can't find the container with id 19fac2d455ee9c52524521a4e3e529d3d957e191c169f967e5b5ebced197931b
Jul 23 03:55:40.979780 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.979762    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zsl88"
Jul 23 03:55:40.989471 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:40.989449    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77281df7_2da2_4a68_a3dc_4b6b9f1d2455.slice/crio-475cbebbde954f8be7eb9e166bbae1e42aa6e9ab52d4f587d84e55cf443c4a20 WatchSource:0}: Error finding container 475cbebbde954f8be7eb9e166bbae1e42aa6e9ab52d4f587d84e55cf443c4a20: Status 404 returned error can't find the container with id 475cbebbde954f8be7eb9e166bbae1e42aa6e9ab52d4f587d84e55cf443c4a20
Jul 23 03:55:40.995254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:40.995237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p47ct"
Jul 23 03:55:41.003295 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:41.003274    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbe5046_a390_4657_968b_f2943a0213b9.slice/crio-b1ce41ba7533716dfcad984d781430a1cec381240d73a1535e8fa7db9d17c634 WatchSource:0}: Error finding container b1ce41ba7533716dfcad984d781430a1cec381240d73a1535e8fa7db9d17c634: Status 404 returned error can't find the container with id b1ce41ba7533716dfcad984d781430a1cec381240d73a1535e8fa7db9d17c634
Jul 23 03:55:41.009644 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.009628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6"
Jul 23 03:55:41.017876 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:41.017849    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7bbf7b_4941_4be6_9966_b794b4915ff7.slice/crio-e8f597747a5b67a06b542f5a77caa0c93a87977ba4bf597ff4586647e566c244 WatchSource:0}: Error finding container e8f597747a5b67a06b542f5a77caa0c93a87977ba4bf597ff4586647e566c244: Status 404 returned error can't find the container with id e8f597747a5b67a06b542f5a77caa0c93a87977ba4bf597ff4586647e566c244
Jul 23 03:55:41.028876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.028852    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqk29"
Jul 23 03:55:41.037063 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:41.037043    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7d2494_e059_4efb_b6a0_87c444f8b948.slice/crio-d7afdcda5df4906fb43bf4bfe41c10a9931ae6e3caf139c47d82805a6b27b848 WatchSource:0}: Error finding container d7afdcda5df4906fb43bf4bfe41c10a9931ae6e3caf139c47d82805a6b27b848: Status 404 returned error can't find the container with id d7afdcda5df4906fb43bf4bfe41c10a9931ae6e3caf139c47d82805a6b27b848
Jul 23 03:55:41.042154 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.042139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8qdjx"
Jul 23 03:55:41.047107 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.047092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:55:41.051796 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:41.051769    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20894936_ef54_43da_8782_fe2fbdf46bb0.slice/crio-f34fece02b9e45848ff7565e39970880e4a6eb9dcb28e3de3650d31ce14e9b47 WatchSource:0}: Error finding container f34fece02b9e45848ff7565e39970880e4a6eb9dcb28e3de3650d31ce14e9b47: Status 404 returned error can't find the container with id f34fece02b9e45848ff7565e39970880e4a6eb9dcb28e3de3650d31ce14e9b47
Jul 23 03:55:41.055384 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:55:41.055366    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod032d1254_cfe2_47d6_ba4d_80752c4f2ae4.slice/crio-15b194b51c5cd7bd6ee5720ea4caa5a679df93456a73cd2cb65850002a332139 WatchSource:0}: Error finding container 15b194b51c5cd7bd6ee5720ea4caa5a679df93456a73cd2cb65850002a332139: Status 404 returned error can't find the container with id 15b194b51c5cd7bd6ee5720ea4caa5a679df93456a73cd2cb65850002a332139
Jul 23 03:55:41.189485 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.187344    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:41.212928 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.212893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:41.213048 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.213022    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:41.213088 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.213077    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:42.213061914 +0000 UTC m=+2.932593503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:41.313311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.313280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:41.313455 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.313391    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:41.313496 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.313457    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:55:42.313426398 +0000 UTC m=+3.032957988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:41.413753 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.413719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:41.413976 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.413956    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:41.414058 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.413985    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:41.414058 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.413999    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:41.414176 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.414058    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:42.414038282 +0000 UTC m=+3.133569888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:41.428707 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.428679    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:41.666792 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.666638    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 03:50:40 +0000 UTC" deadline="2028-04-21 20:52:40.715209134 +0000 UTC"
Jul 23 03:55:41.666792 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.666698    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15328h56m59.048514976s"
Jul 23 03:55:41.761512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.761460    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:41.761693 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.761612    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:41.762132 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.762110    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:41.762252 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.762222    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:41.762341 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.762295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:41.762399 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:41.762365    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:41.787131 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.787074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-798dd" event={"ID":"17528e33-d6b3-4010-9888-08949fa8f635","Type":"ContainerStarted","Data":"d37a06366b3c147c1edf290f45346437a952e98a0d48d9adbcd709a3a636bcfa"}
Jul 23 03:55:41.795805 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.795740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rcdxk" event={"ID":"b3e165c3-a86b-4cd8-808f-67726369adb2","Type":"ContainerStarted","Data":"22e1930926cd500da4b8aaefd63428a72bed36008fd3d29deb486b8de2d0505b"}
Jul 23 03:55:41.809377 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.809344    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8qdjx" event={"ID":"20894936-ef54-43da-8782-fe2fbdf46bb0","Type":"ContainerStarted","Data":"f34fece02b9e45848ff7565e39970880e4a6eb9dcb28e3de3650d31ce14e9b47"}
Jul 23 03:55:41.831895 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.831825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal" event={"ID":"bb1187c486ec4bc41761dae0bcf5f62e","Type":"ContainerStarted","Data":"d4998831a7fbbf93d95978887c52f861b8fa68ad400eecf8cecf9668a21bb7d6"}
Jul 23 03:55:41.857092 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.857061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal" event={"ID":"df25c8e9f4a25a3267946e74f3761d4c","Type":"ContainerStarted","Data":"3de65e20fc0fdee051238f90f4fe1f8d8d6eca2cd004602111512f39d93e85cf"}
Jul 23 03:55:41.870903 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.870809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"15b194b51c5cd7bd6ee5720ea4caa5a679df93456a73cd2cb65850002a332139"}
Jul 23 03:55:41.885283 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.885202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqk29" event={"ID":"cb7d2494-e059-4efb-b6a0-87c444f8b948","Type":"ContainerStarted","Data":"d7afdcda5df4906fb43bf4bfe41c10a9931ae6e3caf139c47d82805a6b27b848"}
Jul 23 03:55:41.888834 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.888750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6" event={"ID":"aa7bbf7b-4941-4be6-9966-b794b4915ff7","Type":"ContainerStarted","Data":"e8f597747a5b67a06b542f5a77caa0c93a87977ba4bf597ff4586647e566c244"}
Jul 23 03:55:41.907483 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.907422    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p47ct" event={"ID":"1bbe5046-a390-4657-968b-f2943a0213b9","Type":"ContainerStarted","Data":"b1ce41ba7533716dfcad984d781430a1cec381240d73a1535e8fa7db9d17c634"}
Jul 23 03:55:41.922519 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.922438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zsl88" event={"ID":"77281df7-2da2-4a68-a3dc-4b6b9f1d2455","Type":"ContainerStarted","Data":"475cbebbde954f8be7eb9e166bbae1e42aa6e9ab52d4f587d84e55cf443c4a20"}
Jul 23 03:55:41.944681 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:41.944567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerStarted","Data":"19fac2d455ee9c52524521a4e3e529d3d957e191c169f967e5b5ebced197931b"}
Jul 23 03:55:42.223356 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:42.222762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:42.223356 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.222937    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:42.223356 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.222996    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:44.22297975 +0000 UTC m=+4.942511341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:42.323839 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:42.323786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:42.324027 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.323959    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:42.324027 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.324021    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:55:44.324002655 +0000 UTC m=+5.043534247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:42.424788 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:42.424128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:42.424788 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.424322    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:42.424788 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.424341    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:42.424788 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.424353    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:42.424788 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:42.424429    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:44.424391518 +0000 UTC m=+5.143923109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:42.666950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:42.666832    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 03:50:40 +0000 UTC" deadline="2028-03-09 14:13:17.120419133 +0000 UTC"
Jul 23 03:55:42.666950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:42.666869    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14290h17m34.453555161s"
Jul 23 03:55:43.526874 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:43.526782    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:43.748549 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:43.748519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:43.748960 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:43.748653    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:43.749039 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:43.749023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:43.749144 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:43.749121    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:43.750287 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:43.750261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:43.750401 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:43.750376    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:44.238531 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:44.238482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:44.238708 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.238661    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:44.238769 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.238724    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:48.238706684 +0000 UTC m=+8.958238277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:44.339880 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:44.339286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:44.339880 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.339446    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:44.339880 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.339516    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:55:48.339489448 +0000 UTC m=+9.059021043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:44.440882 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:44.440126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:44.440882 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.440369    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:44.440882 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.440390    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:44.440882 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.440403    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:44.440882 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:44.440483    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:48.440464868 +0000 UTC m=+9.159996459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:45.750499 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:45.749611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:45.750499 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:45.749949    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:45.750499 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:45.750022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:45.750499 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:45.750159    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:45.750499 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:45.750227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:45.750499 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:45.750299    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:47.746219 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:47.745540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:47.746219 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:47.745582    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:47.746219 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:47.745683    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:47.746219 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:47.745777    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:47.746219 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:47.745815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:47.746219 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:47.745881    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:48.273341 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:48.273244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:48.273545 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.273424    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:48.273545 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.273498    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:56.273478802 +0000 UTC m=+16.993010403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:48.373894 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:48.373856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:48.374076 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.374020    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:48.374140 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.374084    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:55:56.374063419 +0000 UTC m=+17.093595010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:48.474862 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:48.474776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:48.475043 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.474945    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:48.475043 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.474967    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:48.475043 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.474980    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:48.475043 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:48.475041    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:56.475025226 +0000 UTC m=+17.194556831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:49.745977 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:49.745944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:49.746439 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:49.746057    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:49.746439 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:49.746145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:49.746439 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:49.746256    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:49.746439 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:49.746286    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:49.746439 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:49.746346    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:51.747256 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:51.747223    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:51.747712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:51.747263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:51.747712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:51.747317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:51.747712 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:51.747330    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:51.747712 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:51.747382    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:51.747712 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:51.747489    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:53.745395 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:53.744893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:53.745395 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:53.744900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:53.745395 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:53.745040    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:53.745395 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:53.745128    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:53.745395 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:53.745170    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:53.745395 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:53.745283    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:55.744731 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:55.744683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:55.745128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:55.744692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:55.745128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:55.744828    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:55.745128 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:55.744819    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:55.745128 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:55.744924    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:55.745128 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:55.745004    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:56.332319 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:56.332236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:56.332574 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.332362    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:56.332574 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.332453    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:12.332429708 +0000 UTC m=+33.051961316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:56.433639 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:56.433563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:56.433798 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.433688    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:56.433798 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.433752    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:56:12.433736992 +0000 UTC m=+33.153268584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:56.534550 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:56.534512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:56.534741 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.534714    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:56.534741 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.534744    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:56.534931 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.534756    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:56.534931 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:56.534818    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:12.534799804 +0000 UTC m=+33.254331405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:57.744504 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:57.744470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:57.744967 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:57.744478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:57.744967 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:57.744604    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:57.744967 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:57.744698    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:55:57.744967 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:57.744481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:57.744967 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:57.744785    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:59.745390 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:59.745357    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:55:59.745809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:59.745451    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:55:59.745809 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:59.745452    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:55:59.745809 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:59.745564    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:55:59.745809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:55:59.745588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:55:59.745809 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:55:59.745680    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:01.744361 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:01.744274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:01.744751 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:01.744432    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:01.744751 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:01.744485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:01.744751 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:01.744610    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:01.744751 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:01.744657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:01.744751 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:01.744747    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:02.001375 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.001198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"6a25e1c729a30394650cfb6aec3ff27ffe0dd591a7a8fb6e0ddcd5c6836e27c4"}
Jul 23 03:56:02.001375 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.001277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"5896202dffee62b4061f287cc01b116ae0a597d9b84e475bd8880e9dce98d0c5"}
Jul 23 03:56:02.003324 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.003294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zsl88" event={"ID":"77281df7-2da2-4a68-a3dc-4b6b9f1d2455","Type":"ContainerStarted","Data":"fac06b55e83e1f1379c484b4258433279cce5da3c677e9c423813ce25a88f9bc"}
Jul 23 03:56:02.006100 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.006068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-798dd" event={"ID":"17528e33-d6b3-4010-9888-08949fa8f635","Type":"ContainerStarted","Data":"153eed2c410db774edd7469cd6b521be3918788f539bd1123bfc27bb97101555"}
Jul 23 03:56:02.008587 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.008556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal" event={"ID":"df25c8e9f4a25a3267946e74f3761d4c","Type":"ContainerStarted","Data":"595e960bc54d62e1981d0f6f54cc7b4dfcd44fb98602df248a969be50eb1b20d"}
Jul 23 03:56:02.024001 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.023661    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zsl88" podStartSLOduration=2.380688894 podStartE2EDuration="23.02364628s" totalImagesPullingTime="20.642957386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:02.020883941 +0000 UTC m=+22.740415553" watchObservedRunningTime="2026-07-23 03:56:02.02364628 +0000 UTC m=+22.743177889"
Jul 23 03:56:02.037911 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.037676    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-798dd" podStartSLOduration=2.41637382 podStartE2EDuration="23.037657896s" totalImagesPullingTime="20.621284076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:02.037656226 +0000 UTC m=+22.757187836" watchObservedRunningTime="2026-07-23 03:56:02.037657896 +0000 UTC m=+22.757189507"
Jul 23 03:56:02.050906 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:02.050855    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-0.ec2.internal" podStartSLOduration=22.050834299999998 podStartE2EDuration="22.0508343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:02.050545303 +0000 UTC m=+22.770076921" watchObservedRunningTime="2026-07-23 03:56:02.0508343 +0000 UTC m=+22.770365914"
Jul 23 03:56:03.012357 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.012145    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c5a4f1-0afb-4208-b1a8-849405b3eea8" containerID="992dcefb6c430494151e681b14760de198f368776a6ecca6fbef6be57c24086b" exitCode=0
Jul 23 03:56:03.012357 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.012234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerDied","Data":"992dcefb6c430494151e681b14760de198f368776a6ecca6fbef6be57c24086b"}
Jul 23 03:56:03.022626 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.022602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rcdxk" event={"ID":"b3e165c3-a86b-4cd8-808f-67726369adb2","Type":"ContainerStarted","Data":"47631cf0b6941239957336563cdeee0cb02208ca87a1b02f30ec1762e1ed0eae"}
Jul 23 03:56:03.023989 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.023968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8qdjx" event={"ID":"20894936-ef54-43da-8782-fe2fbdf46bb0","Type":"ContainerStarted","Data":"afe94267e84d134d732fd37269cbe3811f2ada24a5a84f6a46e5e810f221842f"}
Jul 23 03:56:03.025366 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.025347    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb1187c486ec4bc41761dae0bcf5f62e" containerID="0476c949f1f3bcbee5137feb370069b682eb5b6ea84375ebc1054b5046e8a16f" exitCode=0
Jul 23 03:56:03.025472 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.025428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal" event={"ID":"bb1187c486ec4bc41761dae0bcf5f62e","Type":"ContainerDied","Data":"0476c949f1f3bcbee5137feb370069b682eb5b6ea84375ebc1054b5046e8a16f"}
Jul 23 03:56:03.027779 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.027765    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 03:56:03.028078 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.028062    2563 generic.go:356] "Generic (PLEG): container finished" podID="032d1254-cfe2-47d6-ba4d-80752c4f2ae4" containerID="6a25e1c729a30394650cfb6aec3ff27ffe0dd591a7a8fb6e0ddcd5c6836e27c4" exitCode=1
Jul 23 03:56:03.028134 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.028120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerDied","Data":"6a25e1c729a30394650cfb6aec3ff27ffe0dd591a7a8fb6e0ddcd5c6836e27c4"}
Jul 23 03:56:03.028197 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.028174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"a6e8886f47bfa6f3b06281b90d9b66531bb753a1b53da6af123cd1acffa09a60"}
Jul 23 03:56:03.028316 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.028217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"335ff9ff4f8a3f97730c55af64d7976465462d6cffde4679f0183940043d0c39"}
Jul 23 03:56:03.028316 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.028256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"a9394e2b937827a03d079fb8850bd75b2f44178447d0aff51929d8b2f2d0ea93"}
Jul 23 03:56:03.028316 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.028284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"092bb7fce9b57705416a07f7c0965a4facd0db8c3b86cd01994975cc46823e78"}
Jul 23 03:56:03.029280 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.029260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqk29" event={"ID":"cb7d2494-e059-4efb-b6a0-87c444f8b948","Type":"ContainerStarted","Data":"2a2a56c2ff36eac9f04fe0f02dcef259bc942788dae8d5d84a84d28e461d656c"}
Jul 23 03:56:03.030511 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.030493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6" event={"ID":"aa7bbf7b-4941-4be6-9966-b794b4915ff7","Type":"ContainerStarted","Data":"b574541f59c7e07827ca6deb8dfa47f7765c7d1f63cb799404d2ace626f1eaf7"}
Jul 23 03:56:03.031871 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.031691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p47ct" event={"ID":"1bbe5046-a390-4657-968b-f2943a0213b9","Type":"ContainerStarted","Data":"2ccb0353a0d619224f52e251d64315326b04d37980a508d687d97fae7e14f9c0"}
Jul 23 03:56:03.055415 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.055377    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rcdxk" podStartSLOduration=3.45712178 podStartE2EDuration="24.055364664s" totalImagesPullingTime="20.598242884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:03.055001886 +0000 UTC m=+23.774533535" watchObservedRunningTime="2026-07-23 03:56:03.055364664 +0000 UTC m=+23.774896274"
Jul 23 03:56:03.069089 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.069057    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zqk29" podStartSLOduration=3.562758142 podStartE2EDuration="24.069047219s" totalImagesPullingTime="20.506289077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:03.069011905 +0000 UTC m=+23.788543493" watchObservedRunningTime="2026-07-23 03:56:03.069047219 +0000 UTC m=+23.788578819"
Jul 23 03:56:03.083823 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.083791    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8qdjx" podStartSLOduration=3.579968659 podStartE2EDuration="24.083780266s" totalImagesPullingTime="20.503811607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:03.08361961 +0000 UTC m=+23.803151219" watchObservedRunningTime="2026-07-23 03:56:03.083780266 +0000 UTC m=+23.803311875"
Jul 23 03:56:03.097389 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.097361    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p47ct" podStartSLOduration=3.524259305 podStartE2EDuration="24.097351435s" totalImagesPullingTime="20.57309213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:03.09733215 +0000 UTC m=+23.816863759" watchObservedRunningTime="2026-07-23 03:56:03.097351435 +0000 UTC m=+23.816883035"
Jul 23 03:56:03.744720 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.744693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:03.744884 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.744713    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:03.744884 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:03.744823    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:03.744884 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.744868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:03.745055 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:03.744971    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:03.745115 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:03.745068    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:03.748474 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:03.748453    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 03:56:04.036471 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:04.036372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6" event={"ID":"aa7bbf7b-4941-4be6-9966-b794b4915ff7","Type":"ContainerStarted","Data":"de1ca1cf559a1bcf0fad1a9e17681ef1bc11ccf4719485e2e5f1434e3914aa7b"}
Jul 23 03:56:04.674602 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:04.674494    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T03:56:03.74847055Z","UUID":"d7d2d5d8-63e7-41cd-9f11-5ce9827e0e5f","Handler":null,"Name":"","Endpoint":""}
Jul 23 03:56:04.676749 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:04.676724    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 03:56:04.676874 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:04.676773    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 03:56:05.041030 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.040994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal" event={"ID":"bb1187c486ec4bc41761dae0bcf5f62e","Type":"ContainerStarted","Data":"cc535c12705f0deac64b2b6c998e8415db2ace41a043a27f3dba7f93fc925061"}
Jul 23 03:56:05.045048 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.045025    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 03:56:05.045392 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.045364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"69f93bebb213740da063071c2e3b075703d9c8a3474b09f3605c3b1f4e09423a"}
Jul 23 03:56:05.056621 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.056571    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-0.ec2.internal" podStartSLOduration=25.056552122 podStartE2EDuration="25.056552122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:05.056198596 +0000 UTC m=+25.775730207" watchObservedRunningTime="2026-07-23 03:56:05.056552122 +0000 UTC m=+25.776083733"
Jul 23 03:56:05.744663 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.744628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:05.744830 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.744628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:05.744830 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:05.744747    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:05.744919 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:05.744855    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:05.744919 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:05.744628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:05.745001 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:05.744959    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:06.050193 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:06.050112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6" event={"ID":"aa7bbf7b-4941-4be6-9966-b794b4915ff7","Type":"ContainerStarted","Data":"f6ba07331cfdfaed58fae445e62754b6038335c9cffd43b4493b1e78dacf2df0"}
Jul 23 03:56:06.071931 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:06.071873    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd2b6" podStartSLOduration=4.214675914 podStartE2EDuration="27.071859536s" totalImagesPullingTime="22.857183622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:06.071474276 +0000 UTC m=+26.791005887" watchObservedRunningTime="2026-07-23 03:56:06.071859536 +0000 UTC m=+26.791391145"
Jul 23 03:56:06.997264 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:06.997194    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqk29_cb7d2494-e059-4efb-b6a0-87c444f8b948/dns-node-resolver/0.log"
Jul 23 03:56:07.209734 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:07.209707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:56:07.210287 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:07.210269    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:56:07.744525 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:07.744321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:07.744636 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:07.744323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:07.744636 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:07.744616    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:07.744758 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:07.744323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:07.744758 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:07.744643    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:07.744758 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:07.744740    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:07.978950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:07.978890    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p47ct_1bbe5046-a390-4657-968b-f2943a0213b9/node-ca/0.log"
Jul 23 03:56:08.056079 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.056053    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c5a4f1-0afb-4208-b1a8-849405b3eea8" containerID="4a76897d75f241bc23f0f302e3b29e2d7923f9d25add69327f85ffb1b7e017b1" exitCode=0
Jul 23 03:56:08.056248 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.056145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerDied","Data":"4a76897d75f241bc23f0f302e3b29e2d7923f9d25add69327f85ffb1b7e017b1"}
Jul 23 03:56:08.059545 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.059526    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 03:56:08.059905 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.059874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"b47c98764045f3a315621f94293ae7591198530f1aeae4fd94768fdd2970b884"}
Jul 23 03:56:08.060125 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.060105    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:56:08.060250 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.060204    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:56:08.060309 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.060258    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:56:08.060309 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.060291    2563 scope.go:117] "RemoveContainer" containerID="6a25e1c729a30394650cfb6aec3ff27ffe0dd591a7a8fb6e0ddcd5c6836e27c4"
Jul 23 03:56:08.060309 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.060302    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:56:08.060801 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.060786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rcdxk"
Jul 23 03:56:08.076154 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.076133    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:56:08.076554 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:08.076536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:56:09.065648 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.065618    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 03:56:09.066650 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.066025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" event={"ID":"032d1254-cfe2-47d6-ba4d-80752c4f2ae4","Type":"ContainerStarted","Data":"9ab391244f3e0d28a68f8cd0e4052f121916e4b2fcfc1d998b8ed9804fd793c5"}
Jul 23 03:56:09.098225 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.098183    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf" podStartSLOduration=9.536616071 podStartE2EDuration="30.098167269s" totalImagesPullingTime="20.561551198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:09.096824991 +0000 UTC m=+29.816356601" watchObservedRunningTime="2026-07-23 03:56:09.098167269 +0000 UTC m=+29.817698925"
Jul 23 03:56:09.744974 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.744814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:09.745132 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.744904    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:09.745132 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:09.745046    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:09.745202 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:09.745151    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:09.745202 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.744938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:09.745261 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:09.745242    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:09.819133 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.819103    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6zfrb"]
Jul 23 03:56:09.823497 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.823472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9k5f"]
Jul 23 03:56:09.827572 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:09.827550    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9cwq"]
Jul 23 03:56:10.070295 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:10.070211    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c5a4f1-0afb-4208-b1a8-849405b3eea8" containerID="bd083bb651ac609a31e016ec66108f7eefc5ea2b75c1dfa130f1a39311aa2259" exitCode=0
Jul 23 03:56:10.070682 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:10.070311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerDied","Data":"bd083bb651ac609a31e016ec66108f7eefc5ea2b75c1dfa130f1a39311aa2259"}
Jul 23 03:56:10.070682 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:10.070630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:10.070776 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:10.070751    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:10.070933 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:10.070902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:10.071022 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:10.071008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:10.071078 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:10.071019    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:10.071132 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:10.071077    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:11.745208 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:11.745179    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:11.745657 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:11.745300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:11.745657 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:11.745313    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:11.745657 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:11.745436    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:11.745657 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:11.745493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:11.745657 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:11.745596    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:12.077341 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:12.077310    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c5a4f1-0afb-4208-b1a8-849405b3eea8" containerID="c163260cb911d557de7f293f90410fd8cb0fe4f8036a94db67f0f7f92b730a76" exitCode=0
Jul 23 03:56:12.077507 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:12.077364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerDied","Data":"c163260cb911d557de7f293f90410fd8cb0fe4f8036a94db67f0f7f92b730a76"}
Jul 23 03:56:12.350633 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:12.350539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:12.350772 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.350662    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:56:12.350772 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.350710    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret podName:4052be7b-0b50-40e8-a250-3e251a80d7d9 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:44.350698073 +0000 UTC m=+65.070229661 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret") pod "global-pull-secret-syncer-z9cwq" (UID: "4052be7b-0b50-40e8-a250-3e251a80d7d9") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:56:12.451520 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:12.451483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:12.451656 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.451618    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:56:12.451695 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.451676    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs podName:17763bad-e4bc-46cb-bd38-dcbb27240c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:56:44.451661966 +0000 UTC m=+65.171193555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs") pod "network-metrics-daemon-6zfrb" (UID: "17763bad-e4bc-46cb-bd38-dcbb27240c1f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:56:12.552173 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:12.552137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:12.552397 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.552381    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:56:12.552489 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.552421    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:56:12.552489 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.552435    2563 projected.go:196] Error preparing data for projected volume kube-api-access-tgs6f for pod openshift-network-diagnostics/network-check-target-q9k5f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:56:12.552583 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:12.552494    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f podName:bd539a0e-afe7-4f62-82e1-e334bfb6dd83 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:44.552476503 +0000 UTC m=+65.272008094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgs6f" (UniqueName: "kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f") pod "network-check-target-q9k5f" (UID: "bd539a0e-afe7-4f62-82e1-e334bfb6dd83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:56:13.744247 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:13.744213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:13.744647 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:13.744335    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:13.744647 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:13.744516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:13.744647 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:13.744521    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:13.744774 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:13.744604    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:13.744774 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:13.744688    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:15.744893 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:15.744693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:15.745367 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:15.744709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:15.745367 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:15.745002    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:15.745367 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:15.744723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:15.745367 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:15.745106    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:15.745367 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:15.745173    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:17.092085 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:17.092034    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c5a4f1-0afb-4208-b1a8-849405b3eea8" containerID="d55edc046b992896352a502a25937f304ae1a2443f0514f4fe27218bea29b7a2" exitCode=0
Jul 23 03:56:17.092571 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:17.092101    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerDied","Data":"d55edc046b992896352a502a25937f304ae1a2443f0514f4fe27218bea29b7a2"}
Jul 23 03:56:17.744965 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:17.744931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:17.745157 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:17.744938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:17.745157 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:17.745067    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:17.745157 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:17.745087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:17.745323 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:17.745215    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:17.745370 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:17.745317    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:18.097128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:18.097044    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c5a4f1-0afb-4208-b1a8-849405b3eea8" containerID="fddadfdb1da97dcba56a247a245f4ee8f9b627ac9c3446c29f4b49f859a67ec8" exitCode=0
Jul 23 03:56:18.097128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:18.097095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerDied","Data":"fddadfdb1da97dcba56a247a245f4ee8f9b627ac9c3446c29f4b49f859a67ec8"}
Jul 23 03:56:19.101929 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:19.101889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-78qc6" event={"ID":"31c5a4f1-0afb-4208-b1a8-849405b3eea8","Type":"ContainerStarted","Data":"6d2b060c752a108f52f3b928c959b17588ff66fdccf8575b681caee26665332f"}
Jul 23 03:56:19.122533 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:19.122485    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-78qc6" podStartSLOduration=8.440783865 podStartE2EDuration="40.122470651s" totalImagesPullingTime="31.681686786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:19.121981714 +0000 UTC m=+39.841513314" watchObservedRunningTime="2026-07-23 03:56:19.122470651 +0000 UTC m=+39.842002260"
Jul 23 03:56:19.746424 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:19.746380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:19.746610 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:19.746447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:19.746610 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:19.746510    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:19.746610 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:19.746528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:19.746778 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:19.746641    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:19.746778 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:19.746730    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:21.744836 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:21.744804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:21.745335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:21.744805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:21.745335 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:21.744905    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:21.745335 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:21.744934    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:21.745335 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:21.745049    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:21.745335 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:21.745087    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:23.747240 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:23.747199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:23.747240 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:23.747214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:23.747240 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:23.747228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:23.747721 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:23.747319    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:23.747721 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:23.747371    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:23.747721 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:23.747449    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:25.744219 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:25.744183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:25.744820 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:25.744197    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:25.744820 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:25.744290    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:25.744820 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:25.744372    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:25.744820 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:25.744198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:25.744820 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:25.744459    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:27.744961 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:27.744928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:27.745468 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:27.744928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:27.745468 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:27.745046    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:27.745468 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:27.744931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:27.745468 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:27.745110    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:27.745468 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:27.745173    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:29.745471 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:29.745434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:29.745928 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:29.745511    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9k5f" podUID="bd539a0e-afe7-4f62-82e1-e334bfb6dd83"
Jul 23 03:56:29.745928 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:29.745528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:29.745928 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:29.745587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:29.745928 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:29.745671    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6zfrb" podUID="17763bad-e4bc-46cb-bd38-dcbb27240c1f"
Jul 23 03:56:29.745928 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:29.745702    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9cwq" podUID="4052be7b-0b50-40e8-a250-3e251a80d7d9"
Jul 23 03:56:30.653444 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.653352    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-0.ec2.internal" event="NodeReady"
Jul 23 03:56:30.653583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.653498    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 03:56:30.706246 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.706211    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w64dx"]
Jul 23 03:56:30.720587 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.720553    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7xd2k"]
Jul 23 03:56:30.720862 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.720717    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.723004 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.722984    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jzncz\""
Jul 23 03:56:30.723122 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.723040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 03:56:30.723712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.723691    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 03:56:30.734605 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.734585    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w64dx"]
Jul 23 03:56:30.734744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.734620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7xd2k"]
Jul 23 03:56:30.734744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.734701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.737183 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.737157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd7d8\""
Jul 23 03:56:30.737183 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.737173    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 03:56:30.737183 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.737162    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 03:56:30.737426 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.737173    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 03:56:30.737426 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.737390    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 03:56:30.794687 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eabeab1-d5db-4b36-83cd-c72e39063356-data-volume\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7eabeab1-d5db-4b36-83cd-c72e39063356-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-tmp-dir\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7eabeab1-d5db-4b36-83cd-c72e39063356-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-config-volume\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-metrics-tls\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kglj\" (UniqueName: \"kubernetes.io/projected/7eabeab1-d5db-4b36-83cd-c72e39063356-kube-api-access-8kglj\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtjwp\" (UniqueName: \"kubernetes.io/projected/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-kube-api-access-jtjwp\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.795072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.794953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7eabeab1-d5db-4b36-83cd-c72e39063356-crio-socket\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.816388 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.816358    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t49dh"]
Jul 23 03:56:30.838648 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.838620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t49dh"]
Jul 23 03:56:30.838782 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.838732    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:30.841481 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.841461    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 03:56:30.841481 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.841478    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 03:56:30.841654 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.841501    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4p5f\""
Jul 23 03:56:30.841654 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.841603    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 03:56:30.895758 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7eabeab1-d5db-4b36-83cd-c72e39063356-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.895758 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-tmp-dir\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7eabeab1-d5db-4b36-83cd-c72e39063356-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d7b224c-f8af-464c-8239-3cb49da00aac-cert\") pod \"ingress-canary-t49dh\" (UID: \"6d7b224c-f8af-464c-8239-3cb49da00aac\") " pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-config-volume\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-metrics-tls\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kglj\" (UniqueName: \"kubernetes.io/projected/7eabeab1-d5db-4b36-83cd-c72e39063356-kube-api-access-8kglj\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrq22\" (UniqueName: \"kubernetes.io/projected/6d7b224c-f8af-464c-8239-3cb49da00aac-kube-api-access-lrq22\") pod \"ingress-canary-t49dh\" (UID: \"6d7b224c-f8af-464c-8239-3cb49da00aac\") " pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtjwp\" (UniqueName: \"kubernetes.io/projected/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-kube-api-access-jtjwp\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.895998 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.895966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7eabeab1-d5db-4b36-83cd-c72e39063356-crio-socket\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.896347 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.896095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eabeab1-d5db-4b36-83cd-c72e39063356-data-volume\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.896347 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.896209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-tmp-dir\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.896485 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.896375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7eabeab1-d5db-4b36-83cd-c72e39063356-crio-socket\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.896485 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.896463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7eabeab1-d5db-4b36-83cd-c72e39063356-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.896594 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.896465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eabeab1-d5db-4b36-83cd-c72e39063356-data-volume\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.896767 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.896746    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-config-volume\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.899952 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.899931    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7eabeab1-d5db-4b36-83cd-c72e39063356-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.900037 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.899937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-metrics-tls\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.904391 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.904346    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtjwp\" (UniqueName: \"kubernetes.io/projected/b08fa11c-9b18-4993-8cc8-cb94dbf83c2a-kube-api-access-jtjwp\") pod \"dns-default-w64dx\" (UID: \"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a\") " pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:30.905571 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.905554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kglj\" (UniqueName: \"kubernetes.io/projected/7eabeab1-d5db-4b36-83cd-c72e39063356-kube-api-access-8kglj\") pod \"insights-runtime-extractor-7xd2k\" (UID: \"7eabeab1-d5db-4b36-83cd-c72e39063356\") " pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:30.997388 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.997349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d7b224c-f8af-464c-8239-3cb49da00aac-cert\") pod \"ingress-canary-t49dh\" (UID: \"6d7b224c-f8af-464c-8239-3cb49da00aac\") " pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:30.997388 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.997389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrq22\" (UniqueName: \"kubernetes.io/projected/6d7b224c-f8af-464c-8239-3cb49da00aac-kube-api-access-lrq22\") pod \"ingress-canary-t49dh\" (UID: \"6d7b224c-f8af-464c-8239-3cb49da00aac\") " pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:30.999851 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:30.999829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d7b224c-f8af-464c-8239-3cb49da00aac-cert\") pod \"ingress-canary-t49dh\" (UID: \"6d7b224c-f8af-464c-8239-3cb49da00aac\") " pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:31.006252 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.006226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrq22\" (UniqueName: \"kubernetes.io/projected/6d7b224c-f8af-464c-8239-3cb49da00aac-kube-api-access-lrq22\") pod \"ingress-canary-t49dh\" (UID: \"6d7b224c-f8af-464c-8239-3cb49da00aac\") " pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:31.030090 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.030063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:31.043784 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.043758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7xd2k"
Jul 23 03:56:31.148327 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.147608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t49dh"
Jul 23 03:56:31.237288 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.236918    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w64dx"]
Jul 23 03:56:31.254377 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.254353    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7xd2k"]
Jul 23 03:56:31.257596 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:31.257572    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eabeab1_d5db_4b36_83cd_c72e39063356.slice/crio-7bb57fe53997c893cb4ac8be925010f4ce24ccb699bc72b5f1ed8173951454e7 WatchSource:0}: Error finding container 7bb57fe53997c893cb4ac8be925010f4ce24ccb699bc72b5f1ed8173951454e7: Status 404 returned error can't find the container with id 7bb57fe53997c893cb4ac8be925010f4ce24ccb699bc72b5f1ed8173951454e7
Jul 23 03:56:31.295538 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.295509    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t49dh"]
Jul 23 03:56:31.298542 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:31.298513    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7b224c_f8af_464c_8239_3cb49da00aac.slice/crio-97d1b8de36cf5f33f700efe8980ef563ac12c1189b2e7f42d3be72430e07be37 WatchSource:0}: Error finding container 97d1b8de36cf5f33f700efe8980ef563ac12c1189b2e7f42d3be72430e07be37: Status 404 returned error can't find the container with id 97d1b8de36cf5f33f700efe8980ef563ac12c1189b2e7f42d3be72430e07be37
Jul 23 03:56:31.749153 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.749126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:31.749333 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.749271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:31.749637 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.749616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:31.751866 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.751839    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 03:56:31.753786 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.753766    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 03:56:31.754002 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.753986    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 03:56:31.754206 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.754191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zlbwn\""
Jul 23 03:56:31.754397 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.754381    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 03:56:31.754627 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:31.754612    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l57h5\""
Jul 23 03:56:32.142881 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:32.142743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w64dx" event={"ID":"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a","Type":"ContainerStarted","Data":"20a373d5efaae8b7255f1d910c83bfb160bb6bfc74383fd9c6cfff85666300b6"}
Jul 23 03:56:32.145036 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:32.144957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t49dh" event={"ID":"6d7b224c-f8af-464c-8239-3cb49da00aac","Type":"ContainerStarted","Data":"97d1b8de36cf5f33f700efe8980ef563ac12c1189b2e7f42d3be72430e07be37"}
Jul 23 03:56:32.147920 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:32.147813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xd2k" event={"ID":"7eabeab1-d5db-4b36-83cd-c72e39063356","Type":"ContainerStarted","Data":"7bb57fe53997c893cb4ac8be925010f4ce24ccb699bc72b5f1ed8173951454e7"}
Jul 23 03:56:36.665297 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.664918    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"]
Jul 23 03:56:36.680528 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.680499    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"]
Jul 23 03:56:36.680679 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.680609    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:36.683231 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.683197    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-72qf6\""
Jul 23 03:56:36.683367 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.683298    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 23 03:56:36.744626 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.744573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/901bfc22-c9e2-4e5d-b0bb-603cb53daa45-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-25bvg\" (UID: \"901bfc22-c9e2-4e5d-b0bb-603cb53daa45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:36.845336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.845294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/901bfc22-c9e2-4e5d-b0bb-603cb53daa45-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-25bvg\" (UID: \"901bfc22-c9e2-4e5d-b0bb-603cb53daa45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:36.848279 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.848249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/901bfc22-c9e2-4e5d-b0bb-603cb53daa45-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-25bvg\" (UID: \"901bfc22-c9e2-4e5d-b0bb-603cb53daa45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:36.992979 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:36.992937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:37.141796 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:37.141632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"]
Jul 23 03:56:37.164232 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:37.164176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t49dh" event={"ID":"6d7b224c-f8af-464c-8239-3cb49da00aac","Type":"ContainerStarted","Data":"6668863d21774ce361b1a2821579a54e7bd65315a81fc2cd75399eb496951319"}
Jul 23 03:56:37.165702 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:37.165675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w64dx" event={"ID":"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a","Type":"ContainerStarted","Data":"44d0e8e5869516168b4aa7cd073d8f4cc0e9bbdbd0aa83c67b37d2a01f49168c"}
Jul 23 03:56:37.182885 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:37.182829    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t49dh" podStartSLOduration=1.548158366 podStartE2EDuration="7.182810153s" totalImagesPullingTime="5.634651787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:37.182682487 +0000 UTC m=+57.902214100" watchObservedRunningTime="2026-07-23 03:56:37.182810153 +0000 UTC m=+57.902341741"
Jul 23 03:56:37.241426 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:37.241380    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901bfc22_c9e2_4e5d_b0bb_603cb53daa45.slice/crio-f03fb6d147cbf0b6a68a238b71a98566bc6c5d5ecc699b51108fb9bff2b772af WatchSource:0}: Error finding container f03fb6d147cbf0b6a68a238b71a98566bc6c5d5ecc699b51108fb9bff2b772af: Status 404 returned error can't find the container with id f03fb6d147cbf0b6a68a238b71a98566bc6c5d5ecc699b51108fb9bff2b772af
Jul 23 03:56:38.170016 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:38.169985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xd2k" event={"ID":"7eabeab1-d5db-4b36-83cd-c72e39063356","Type":"ContainerStarted","Data":"c11f3f500bd4252fa0a39551239af8e39c39cfc9c7ef5415de40bd4ad1cc590e"}
Jul 23 03:56:38.172365 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:38.172331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w64dx" event={"ID":"b08fa11c-9b18-4993-8cc8-cb94dbf83c2a","Type":"ContainerStarted","Data":"b659705c9cc29f0dc948c4c565d2adec24faf2e413ba7489377dee27e35ced7b"}
Jul 23 03:56:38.173923 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:38.173897    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg" event={"ID":"901bfc22-c9e2-4e5d-b0bb-603cb53daa45","Type":"ContainerStarted","Data":"f03fb6d147cbf0b6a68a238b71a98566bc6c5d5ecc699b51108fb9bff2b772af"}
Jul 23 03:56:38.189847 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:38.189797    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w64dx" podStartSLOduration=2.498628363 podStartE2EDuration="8.189778725s" totalImagesPullingTime="5.691150362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:38.188500468 +0000 UTC m=+58.908032079" watchObservedRunningTime="2026-07-23 03:56:38.189778725 +0000 UTC m=+58.909310336"
Jul 23 03:56:39.179470 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.179404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg" event={"ID":"901bfc22-c9e2-4e5d-b0bb-603cb53daa45","Type":"ContainerStarted","Data":"f27b5e759aaa865b6495e5f6c69d7ecd6f8ec207d93f0aded377595f99863de8"}
Jul 23 03:56:39.179916 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.179709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:39.179916 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.179752    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:39.185023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.184999    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg"
Jul 23 03:56:39.195151 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.195099    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-25bvg" podStartSLOduration=1.9445487049999999 podStartE2EDuration="3.195081425s" totalImagesPullingTime="1.25053272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:39.194233359 +0000 UTC m=+59.913764972" watchObservedRunningTime="2026-07-23 03:56:39.195081425 +0000 UTC m=+59.914613036"
Jul 23 03:56:39.727172 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.727137    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"]
Jul 23 03:56:39.738758 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.738396    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"]
Jul 23 03:56:39.738758 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.738542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.743175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.742160    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rklbk\""
Jul 23 03:56:39.743175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.742170    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 03:56:39.743175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.742200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 03:56:39.743175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.742202    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 03:56:39.743175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.742164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 23 03:56:39.743175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.742818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 23 03:56:39.767570 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.767539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lsvh\" (UniqueName: \"kubernetes.io/projected/0f0ea308-d3c2-4745-aff8-cd7765d594aa-kube-api-access-7lsvh\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.767713 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.767609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f0ea308-d3c2-4745-aff8-cd7765d594aa-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.767782 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.767725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.767782 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.767765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.868524 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.868483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.868679 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.868530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.868679 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.868571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsvh\" (UniqueName: \"kubernetes.io/projected/0f0ea308-d3c2-4745-aff8-cd7765d594aa-kube-api-access-7lsvh\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.868679 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.868618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f0ea308-d3c2-4745-aff8-cd7765d594aa-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.868679 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:39.868660    2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 03:56:39.868848 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:39.868735    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-tls podName:0f0ea308-d3c2-4745-aff8-cd7765d594aa nodeName:}" failed. No retries permitted until 2026-07-23 03:56:40.368713634 +0000 UTC m=+61.088245224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-qfb5s" (UID: "0f0ea308-d3c2-4745-aff8-cd7765d594aa") : secret "prometheus-operator-tls" not found
Jul 23 03:56:39.869337 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.869311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f0ea308-d3c2-4745-aff8-cd7765d594aa-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.871546 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.871519    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:39.878942 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:39.878920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsvh\" (UniqueName: \"kubernetes.io/projected/0f0ea308-d3c2-4745-aff8-cd7765d594aa-kube-api-access-7lsvh\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:40.081688 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:40.081605    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qstf"
Jul 23 03:56:40.373484 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:40.373384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:40.376161 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:40.376127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f0ea308-d3c2-4745-aff8-cd7765d594aa-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qfb5s\" (UID: \"0f0ea308-d3c2-4745-aff8-cd7765d594aa\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:40.650745 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:40.650658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"
Jul 23 03:56:41.266098 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:41.266074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s"]
Jul 23 03:56:41.268961 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:41.268932    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0ea308_d3c2_4745_aff8_cd7765d594aa.slice/crio-0f265db5dd0b789bd5953693bc5aa3395165ae1ff3dde792c7c434d287c2c2f9 WatchSource:0}: Error finding container 0f265db5dd0b789bd5953693bc5aa3395165ae1ff3dde792c7c434d287c2c2f9: Status 404 returned error can't find the container with id 0f265db5dd0b789bd5953693bc5aa3395165ae1ff3dde792c7c434d287c2c2f9
Jul 23 03:56:42.194420 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:42.194361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xd2k" event={"ID":"7eabeab1-d5db-4b36-83cd-c72e39063356","Type":"ContainerStarted","Data":"19c0ad045d98659a8bac8421d355c28103033b13611f0590c88ca2a7c88a4946"}
Jul 23 03:56:42.195662 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:42.195635    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s" event={"ID":"0f0ea308-d3c2-4745-aff8-cd7765d594aa","Type":"ContainerStarted","Data":"0f265db5dd0b789bd5953693bc5aa3395165ae1ff3dde792c7c434d287c2c2f9"}
Jul 23 03:56:43.200593 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:43.200556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s" event={"ID":"0f0ea308-d3c2-4745-aff8-cd7765d594aa","Type":"ContainerStarted","Data":"3d0d025dbc115d234b513ebc3ed53291a08a7d9a0510995f7b7288b2bc6bb9df"}
Jul 23 03:56:43.201008 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:43.200623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s" event={"ID":"0f0ea308-d3c2-4745-aff8-cd7765d594aa","Type":"ContainerStarted","Data":"8954aeb0a9163c47acf7c2d9c8b3ebd733b68d50d50c3a1d123304297250930a"}
Jul 23 03:56:43.219437 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:43.217513    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qfb5s" podStartSLOduration=2.965415338 podStartE2EDuration="4.217493217s" totalImagesPullingTime="1.252077879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:43.216146833 +0000 UTC m=+63.935678438" watchObservedRunningTime="2026-07-23 03:56:43.217493217 +0000 UTC m=+63.937024828"
Jul 23 03:56:44.406295 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.406252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:44.408893 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.408874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 03:56:44.419593 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.419571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4052be7b-0b50-40e8-a250-3e251a80d7d9-original-pull-secret\") pod \"global-pull-secret-syncer-z9cwq\" (UID: \"4052be7b-0b50-40e8-a250-3e251a80d7d9\") " pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:44.506925 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.506888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:44.509463 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.509442    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 03:56:44.519959 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.519927    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17763bad-e4bc-46cb-bd38-dcbb27240c1f-metrics-certs\") pod \"network-metrics-daemon-6zfrb\" (UID: \"17763bad-e4bc-46cb-bd38-dcbb27240c1f\") " pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:44.607786 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.607745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:44.610445 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.610404    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 03:56:44.620974 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.620954    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 03:56:44.631652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.631625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgs6f\" (UniqueName: \"kubernetes.io/projected/bd539a0e-afe7-4f62-82e1-e334bfb6dd83-kube-api-access-tgs6f\") pod \"network-check-target-q9k5f\" (UID: \"bd539a0e-afe7-4f62-82e1-e334bfb6dd83\") " pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:44.664532 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.664442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9cwq"
Jul 23 03:56:44.677392 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.677370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zlbwn\""
Jul 23 03:56:44.682326 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.682305    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l57h5\""
Jul 23 03:56:44.685966 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.685950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:44.690651 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.690633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6zfrb"
Jul 23 03:56:44.901244 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:44.901220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9cwq"]
Jul 23 03:56:44.904001 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:44.903973    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4052be7b_0b50_40e8_a250_3e251a80d7d9.slice/crio-7f2f638718f638ed7983e091946690b9c99c39d4fa22778645541a8e5b6aef71 WatchSource:0}: Error finding container 7f2f638718f638ed7983e091946690b9c99c39d4fa22778645541a8e5b6aef71: Status 404 returned error can't find the container with id 7f2f638718f638ed7983e091946690b9c99c39d4fa22778645541a8e5b6aef71
Jul 23 03:56:45.109629 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.109599    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"]
Jul 23 03:56:45.112829 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.112811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.117268 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.117241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 23 03:56:45.117635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.117241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bm6gz\""
Jul 23 03:56:45.117635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.117241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 23 03:56:45.117635 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.117629    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gz756"]
Jul 23 03:56:45.120568 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.120551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.123051 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.123034    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 03:56:45.123188 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.123165    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9z7rs\""
Jul 23 03:56:45.123188 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.123178    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 03:56:45.123583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.123566    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"]
Jul 23 03:56:45.123770 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.123756    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 03:56:45.132840 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.132713    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"]
Jul 23 03:56:45.134883 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:45.134861    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd539a0e_afe7_4f62_82e1_e334bfb6dd83.slice/crio-8c376ef94ff7e71d10972f3afb2cf0c11b3c819655e5f8890dcc9221302bdc64 WatchSource:0}: Error finding container 8c376ef94ff7e71d10972f3afb2cf0c11b3c819655e5f8890dcc9221302bdc64: Status 404 returned error can't find the container with id 8c376ef94ff7e71d10972f3afb2cf0c11b3c819655e5f8890dcc9221302bdc64
Jul 23 03:56:45.135936 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.135909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9k5f"]
Jul 23 03:56:45.136039 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.135953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6zfrb"]
Jul 23 03:56:45.136039 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:45.135984    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17763bad_e4bc_46cb_bd38_dcbb27240c1f.slice/crio-30367dd15b4b5e879e76efc20aad058d3face337af699110293fd6a9f02e0592 WatchSource:0}: Error finding container 30367dd15b4b5e879e76efc20aad058d3face337af699110293fd6a9f02e0592: Status 404 returned error can't find the container with id 30367dd15b4b5e879e76efc20aad058d3face337af699110293fd6a9f02e0592
Jul 23 03:56:45.136153 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.136052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.138581 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.138362    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 23 03:56:45.138581 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.138519    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 23 03:56:45.138581 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.138530    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-56cvl\""
Jul 23 03:56:45.138871 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.138716    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 23 03:56:45.147051 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.147020    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"]
Jul 23 03:56:45.207980 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.207948    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9k5f" event={"ID":"bd539a0e-afe7-4f62-82e1-e334bfb6dd83","Type":"ContainerStarted","Data":"8c376ef94ff7e71d10972f3afb2cf0c11b3c819655e5f8890dcc9221302bdc64"}
Jul 23 03:56:45.209363 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.209338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6zfrb" event={"ID":"17763bad-e4bc-46cb-bd38-dcbb27240c1f","Type":"ContainerStarted","Data":"30367dd15b4b5e879e76efc20aad058d3face337af699110293fd6a9f02e0592"}
Jul 23 03:56:45.210446 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.210400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9cwq" event={"ID":"4052be7b-0b50-40e8-a250-3e251a80d7d9","Type":"ContainerStarted","Data":"7f2f638718f638ed7983e091946690b9c99c39d4fa22778645541a8e5b6aef71"}
Jul 23 03:56:45.212181 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.212162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xd2k" event={"ID":"7eabeab1-d5db-4b36-83cd-c72e39063356","Type":"ContainerStarted","Data":"5c795b2abd05ba1f433ab15f40e99d3d933af313d86ce3dbdd24b0b27646aeac"}
Jul 23 03:56:45.214693 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.214672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4da9a14-ffa8-4545-8826-343b63133c1f-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.214751 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.214705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.214751 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.214721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjg42\" (UniqueName: \"kubernetes.io/projected/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-api-access-kjg42\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.214860 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.214762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f726308-3e99-4682-933f-13a03681d96a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.214924 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.214787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-sys\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215210 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwgjs\" (UniqueName: \"kubernetes.io/projected/a12afeee-5893-4a93-83d1-f0ada3404c9e-kube-api-access-rwgjs\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215324 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215383 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f726308-3e99-4682-933f-13a03681d96a-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.215471 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-tls\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215471 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215461    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj5sw\" (UniqueName: \"kubernetes.io/projected/3f726308-3e99-4682-933f-13a03681d96a-kube-api-access-dj5sw\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.215568 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.215568 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-root\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215666 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-wtmp\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215666 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a12afeee-5893-4a93-83d1-f0ada3404c9e-metrics-client-ca\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215666 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.215819 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-textfile\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215819 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.215819 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f726308-3e99-4682-933f-13a03681d96a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.215819 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.215806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c4da9a14-ffa8-4545-8826-343b63133c1f-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.316794 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-textfile\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.316794 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f726308-3e99-4682-933f-13a03681d96a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c4da9a14-ffa8-4545-8826-343b63133c1f-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4da9a14-ffa8-4545-8826-343b63133c1f-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjg42\" (UniqueName: \"kubernetes.io/projected/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-api-access-kjg42\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f726308-3e99-4682-933f-13a03681d96a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.316982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-sys\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwgjs\" (UniqueName: \"kubernetes.io/projected/a12afeee-5893-4a93-83d1-f0ada3404c9e-kube-api-access-rwgjs\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f726308-3e99-4682-933f-13a03681d96a-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-tls\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj5sw\" (UniqueName: \"kubernetes.io/projected/3f726308-3e99-4682-933f-13a03681d96a-kube-api-access-dj5sw\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.317196 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-root\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-textfile\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-wtmp\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a12afeee-5893-4a93-83d1-f0ada3404c9e-metrics-client-ca\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-root\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.317644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-sys\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:45.317716    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 23 03:56:45.317876 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:45.317790    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-tls podName:c4da9a14-ffa8-4545-8826-343b63133c1f nodeName:}" failed. No retries permitted until 2026-07-23 03:56:45.817769442 +0000 UTC m=+66.537301033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-x6fqk" (UID: "c4da9a14-ffa8-4545-8826-343b63133c1f") : secret "kube-state-metrics-tls" not found
Jul 23 03:56:45.318557 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.318103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-wtmp\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.318557 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.318182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4da9a14-ffa8-4545-8826-343b63133c1f-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.318885 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.318860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.319033 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.319006    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f726308-3e99-4682-933f-13a03681d96a-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.319115 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.319068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c4da9a14-ffa8-4545-8826-343b63133c1f-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.319115 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.319082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a12afeee-5893-4a93-83d1-f0ada3404c9e-metrics-client-ca\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.319907 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.319890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f726308-3e99-4682-933f-13a03681d96a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.320085 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.320063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-tls\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.320192 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.320178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.320715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.320696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a12afeee-5893-4a93-83d1-f0ada3404c9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.320796 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.320775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f726308-3e99-4682-933f-13a03681d96a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.330337 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.330315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwgjs\" (UniqueName: \"kubernetes.io/projected/a12afeee-5893-4a93-83d1-f0ada3404c9e-kube-api-access-rwgjs\") pod \"node-exporter-gz756\" (UID: \"a12afeee-5893-4a93-83d1-f0ada3404c9e\") " pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.333802 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.333777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj5sw\" (UniqueName: \"kubernetes.io/projected/3f726308-3e99-4682-933f-13a03681d96a-kube-api-access-dj5sw\") pod \"openshift-state-metrics-79f6958cdf-vvzhp\" (UID: \"3f726308-3e99-4682-933f-13a03681d96a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.335647 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.335628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjg42\" (UniqueName: \"kubernetes.io/projected/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-api-access-kjg42\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.422114 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.422035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"
Jul 23 03:56:45.430321 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.430113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gz756"
Jul 23 03:56:45.442824 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:45.442782    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12afeee_5893_4a93_83d1_f0ada3404c9e.slice/crio-ff060b116618f4b6e41699d1d41a81ea0c79d482748c3a6613190d3007425116 WatchSource:0}: Error finding container ff060b116618f4b6e41699d1d41a81ea0c79d482748c3a6613190d3007425116: Status 404 returned error can't find the container with id ff060b116618f4b6e41699d1d41a81ea0c79d482748c3a6613190d3007425116
Jul 23 03:56:45.564214 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.564160    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7xd2k" podStartSLOduration=2.200645567 podStartE2EDuration="15.56413461s" totalImagesPullingTime="13.363489043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:56:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:45.244673329 +0000 UTC m=+65.964204938" watchObservedRunningTime="2026-07-23 03:56:45.56413461 +0000 UTC m=+66.283666223"
Jul 23 03:56:45.564531 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.564514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp"]
Jul 23 03:56:45.566093 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:45.566062    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f726308_3e99_4682_933f_13a03681d96a.slice/crio-5b550ba05b26bf67b38908a61023ead4423f741d7eedcb8f4ef44e661d1c5e48 WatchSource:0}: Error finding container 5b550ba05b26bf67b38908a61023ead4423f741d7eedcb8f4ef44e661d1c5e48: Status 404 returned error can't find the container with id 5b550ba05b26bf67b38908a61023ead4423f741d7eedcb8f4ef44e661d1c5e48
Jul 23 03:56:45.823036 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.822539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:45.828865 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:45.828822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4da9a14-ffa8-4545-8826-343b63133c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-x6fqk\" (UID: \"c4da9a14-ffa8-4545-8826-343b63133c1f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:46.046050 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.046014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"
Jul 23 03:56:46.218301 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.218265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gz756" event={"ID":"a12afeee-5893-4a93-83d1-f0ada3404c9e","Type":"ContainerStarted","Data":"ff060b116618f4b6e41699d1d41a81ea0c79d482748c3a6613190d3007425116"}
Jul 23 03:56:46.221148 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.221115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp" event={"ID":"3f726308-3e99-4682-933f-13a03681d96a","Type":"ContainerStarted","Data":"81741c5f050deff86b4e19351eba8a8a09ba41dbec73e819062042fe18229e8f"}
Jul 23 03:56:46.221294 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.221181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp" event={"ID":"3f726308-3e99-4682-933f-13a03681d96a","Type":"ContainerStarted","Data":"be90b164694d2501427a3540d776610dfdc8990a2e915feb6b735460564b2f71"}
Jul 23 03:56:46.221294 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.221211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp" event={"ID":"3f726308-3e99-4682-933f-13a03681d96a","Type":"ContainerStarted","Data":"5b550ba05b26bf67b38908a61023ead4423f741d7eedcb8f4ef44e661d1c5e48"}
Jul 23 03:56:46.549724 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.549686    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:56:46.555636 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.555155    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.562345 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.561870    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 03:56:46.562345 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562022    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 03:56:46.562345 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562151    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 03:56:46.562345 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562296    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bnjnp\""
Jul 23 03:56:46.562703 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562440    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 03:56:46.562703 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 03:56:46.562810 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562296    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 03:56:46.562928 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 03:56:46.562928 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.562908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 03:56:46.563200 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.563184    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 03:56:46.580712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.580684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:56:46.630662 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.630848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.630848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.630848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trxsk\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-kube-api-access-trxsk\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.630848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.630848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.630848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-web-config\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.631142 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-config-volume\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.631142 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-config-out\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.631142 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.631142 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.631142 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.630987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.631142 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.631033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732067 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732163 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-web-config\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732163 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-config-volume\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732163 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-config-out\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732532 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732532 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732532 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732532 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trxsk\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-kube-api-access-trxsk\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732532 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.732843 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.732823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.734848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.734817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.736790 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.736766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-web-config\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.738829 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.737908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.738829 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.738085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.738829 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.738277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-config-volume\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.738829 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.738788    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.741154 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.740888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.741154 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.741117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-config-out\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.741154 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.741129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.741356 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.741164    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.743242 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.743197    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.751881 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.751831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trxsk\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-kube-api-access-trxsk\") pod \"alertmanager-main-0\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.867529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.867454    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:56:46.899876 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:46.899643    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-x6fqk"]
Jul 23 03:56:47.072814 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.072718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:56:47.076871 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:47.076840    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376ec373_6628_4da1_940a_fd212a8432be.slice/crio-7a7acd3ff0a1e4efc2811896f4ebc1f0937420bc47a774d0340f50be44a4b56e WatchSource:0}: Error finding container 7a7acd3ff0a1e4efc2811896f4ebc1f0937420bc47a774d0340f50be44a4b56e: Status 404 returned error can't find the container with id 7a7acd3ff0a1e4efc2811896f4ebc1f0937420bc47a774d0340f50be44a4b56e
Jul 23 03:56:47.227663 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.227628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"7a7acd3ff0a1e4efc2811896f4ebc1f0937420bc47a774d0340f50be44a4b56e"}
Jul 23 03:56:47.230317 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.230281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6zfrb" event={"ID":"17763bad-e4bc-46cb-bd38-dcbb27240c1f","Type":"ContainerStarted","Data":"2f4f6a6040a7a18fb586dbb29e1ac226c865acbcb0d5184d48f98f05675d0e7c"}
Jul 23 03:56:47.230500 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.230339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6zfrb" event={"ID":"17763bad-e4bc-46cb-bd38-dcbb27240c1f","Type":"ContainerStarted","Data":"7291557a797ee8ab48ef5bbb7554e0a0ee7c5d5529b7154969e13670f6ea645f"}
Jul 23 03:56:47.232885 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.232788    2563 generic.go:356] "Generic (PLEG): container finished" podID="a12afeee-5893-4a93-83d1-f0ada3404c9e" containerID="2f24d937f404173991bcd5e8fcb53b9372c2e7a13674c8ae21c5c2f45499d841" exitCode=0
Jul 23 03:56:47.232885 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.232869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gz756" event={"ID":"a12afeee-5893-4a93-83d1-f0ada3404c9e","Type":"ContainerDied","Data":"2f24d937f404173991bcd5e8fcb53b9372c2e7a13674c8ae21c5c2f45499d841"}
Jul 23 03:56:47.234517 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.234479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk" event={"ID":"c4da9a14-ffa8-4545-8826-343b63133c1f","Type":"ContainerStarted","Data":"1cbd468d0a1281491e9b415f036375eabd2471d466b565246050b982534b3ea0"}
Jul 23 03:56:47.253561 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.253497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6zfrb" podStartSLOduration=66.658381634 podStartE2EDuration="1m8.253476008s" totalImagesPullingTime="1.595094374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:47.25339231 +0000 UTC m=+67.972923921" watchObservedRunningTime="2026-07-23 03:56:47.253476008 +0000 UTC m=+67.973007620"
Jul 23 03:56:47.457443 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.457387    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-744786bd49-t9xps"]
Jul 23 03:56:47.462715 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.462688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.466070 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.465886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 23 03:56:47.466070 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.465886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 23 03:56:47.466254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.465886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 23 03:56:47.466734 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.466563    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 23 03:56:47.466734 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.466623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 23 03:56:47.466966 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.466943    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3alcjbco5s0fq\""
Jul 23 03:56:47.467031 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.466970    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5cmh5\""
Jul 23 03:56:47.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.539773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.539806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.539810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bc3eb25-192a-4840-89b8-912db55cb57b-metrics-client-ca\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.540038 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.539916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.540038 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.539975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.540038 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.540015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-tls\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.540262 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.540053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-grpc-tls\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.540262 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.540089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txwnl\" (UniqueName: \"kubernetes.io/projected/4bc3eb25-192a-4840-89b8-912db55cb57b-kube-api-access-txwnl\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.540262 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.540151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.542254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.542163    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-744786bd49-t9xps"]
Jul 23 03:56:47.640890 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.640867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-tls\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.640909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-grpc-tls\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.640943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txwnl\" (UniqueName: \"kubernetes.io/projected/4bc3eb25-192a-4840-89b8-912db55cb57b-kube-api-access-txwnl\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.640991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.641035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.641059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bc3eb25-192a-4840-89b8-912db55cb57b-metrics-client-ca\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.641115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.641254 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.641150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.642999 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.642706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bc3eb25-192a-4840-89b8-912db55cb57b-metrics-client-ca\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.646166 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.646104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.646166 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.646136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-grpc-tls\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.646789 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.646678    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.647250 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.647217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.647872 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.647816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-tls\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.648499 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.648448    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bc3eb25-192a-4840-89b8-912db55cb57b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.658199 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.658153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txwnl\" (UniqueName: \"kubernetes.io/projected/4bc3eb25-192a-4840-89b8-912db55cb57b-kube-api-access-txwnl\") pod \"thanos-querier-744786bd49-t9xps\" (UID: \"4bc3eb25-192a-4840-89b8-912db55cb57b\") " pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:47.776290 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:47.776246    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:48.242514 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:48.242476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gz756" event={"ID":"a12afeee-5893-4a93-83d1-f0ada3404c9e","Type":"ContainerStarted","Data":"60230c0d99bdb72e47d8a98f754dab982ed25a0c8677b8e26d51d28d6849b243"}
Jul 23 03:56:48.242700 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:48.242542    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gz756" event={"ID":"a12afeee-5893-4a93-83d1-f0ada3404c9e","Type":"ContainerStarted","Data":"7f9cc1c753e488a4d4f8d65ddb38f9fe23c5f8f0184d2ba4735ded6ab8d09d21"}
Jul 23 03:56:48.246149 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:48.246121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp" event={"ID":"3f726308-3e99-4682-933f-13a03681d96a","Type":"ContainerStarted","Data":"afc9531fe13bf98598ad53e72ca3093e5a439a0cf22c94fceb2adb0702c20e0c"}
Jul 23 03:56:48.269874 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:48.265117    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gz756" podStartSLOduration=1.8628100459999999 podStartE2EDuration="3.265095981s" totalImagesPullingTime="1.292582452s" totalInitContainerRuntime="109.703483ms" isStatefulPod=true podCreationTimestamp="2026-07-23 03:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:48.264154969 +0000 UTC m=+68.983686579" watchObservedRunningTime="2026-07-23 03:56:48.265095981 +0000 UTC m=+68.984627592"
Jul 23 03:56:48.288722 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:48.288665    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vvzhp" podStartSLOduration=1.394230207 podStartE2EDuration="3.288645733s" totalImagesPullingTime="1.894415526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:48.287108725 +0000 UTC m=+69.006640335" watchObservedRunningTime="2026-07-23 03:56:48.288645733 +0000 UTC m=+69.008177341"
Jul 23 03:56:49.186286 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.186257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w64dx"
Jul 23 03:56:49.462545 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.462511    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-867b956d86-md2s6"]
Jul 23 03:56:49.482629 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.482602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-867b956d86-md2s6"]
Jul 23 03:56:49.482768 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.482739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.485626 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.485597    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 03:56:49.485626 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.485620    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 03:56:49.485812 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.485627    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 03:56:49.485812 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.485620    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 03:56:49.486505 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.486486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-28wrb\""
Jul 23 03:56:49.486623 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.486494    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cca4iqs4n5r0g\""
Jul 23 03:56:49.565613 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-secret-metrics-server-client-certs\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.565613 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjcj\" (UniqueName: \"kubernetes.io/projected/44408668-1169-4463-a5f3-2721bf8da747-kube-api-access-lsjcj\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.565818 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/44408668-1169-4463-a5f3-2721bf8da747-metrics-server-audit-profiles\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.565818 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/44408668-1169-4463-a5f3-2721bf8da747-audit-log\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.565818 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-secret-metrics-server-tls\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.565818 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44408668-1169-4463-a5f3-2721bf8da747-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.565818 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.565771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-client-ca-bundle\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667115 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-secret-metrics-server-client-certs\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667303 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjcj\" (UniqueName: \"kubernetes.io/projected/44408668-1169-4463-a5f3-2721bf8da747-kube-api-access-lsjcj\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667303 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/44408668-1169-4463-a5f3-2721bf8da747-metrics-server-audit-profiles\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667303 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/44408668-1169-4463-a5f3-2721bf8da747-audit-log\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667303 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-secret-metrics-server-tls\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667303 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44408668-1169-4463-a5f3-2721bf8da747-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667303 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-client-ca-bundle\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.667691 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.667668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/44408668-1169-4463-a5f3-2721bf8da747-audit-log\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.668583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.668506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44408668-1169-4463-a5f3-2721bf8da747-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.668583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.668512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/44408668-1169-4463-a5f3-2721bf8da747-metrics-server-audit-profiles\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.670266 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.670223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-secret-metrics-server-tls\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.670266 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.670243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-client-ca-bundle\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.670743 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.670724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/44408668-1169-4463-a5f3-2721bf8da747-secret-metrics-server-client-certs\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.676695 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.676675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjcj\" (UniqueName: \"kubernetes.io/projected/44408668-1169-4463-a5f3-2721bf8da747-kube-api-access-lsjcj\") pod \"metrics-server-867b956d86-md2s6\" (UID: \"44408668-1169-4463-a5f3-2721bf8da747\") " pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.791018 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.790941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:56:49.869722 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.869687    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"]
Jul 23 03:56:49.891941 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.891899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"]
Jul 23 03:56:49.892115 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.892014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:49.894719 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.894670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dqqqr\""
Jul 23 03:56:49.894719 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.894702    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 23 03:56:49.970435 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.970380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlx47\" (UniqueName: \"kubernetes.io/projected/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-kube-api-access-xlx47\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:49.970603 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:49.970540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:50.071932 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.071830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:50.071932 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.071917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlx47\" (UniqueName: \"kubernetes.io/projected/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-kube-api-access-xlx47\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:50.072153 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:50.072011    2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 23 03:56:50.072215 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:56:50.072163    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-monitoring-plugin-cert podName:6f2ad05c-2bd6-45f9-8c44-8b4a3b839606 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:50.572138518 +0000 UTC m=+71.291670107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-xkvrq" (UID: "6f2ad05c-2bd6-45f9-8c44-8b4a3b839606") : secret "monitoring-plugin-cert" not found
Jul 23 03:56:50.082381 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.082357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlx47\" (UniqueName: \"kubernetes.io/projected/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-kube-api-access-xlx47\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:50.373095 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.373011    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b84dcb675-lm529"]
Jul 23 03:56:50.376337 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.376315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.379088 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.378950    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 23 03:56:50.379088 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.379070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 23 03:56:50.379251 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.379203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 23 03:56:50.379251 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.379221    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 23 03:56:50.379360 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.379295    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5hk6b\""
Jul 23 03:56:50.379360 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.379309    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 23 03:56:50.385432 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.385333    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 23 03:56:50.391323 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.391299    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b84dcb675-lm529"]
Jul 23 03:56:50.475622 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475622 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-federate-client-tls\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-secret-telemeter-client\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475848 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475821    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-metrics-client-ca\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475982 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-serving-certs-ca-bundle\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475982 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2p8x\" (UniqueName: \"kubernetes.io/projected/14ddbcfc-db2b-433d-a8a0-f43adce481de-kube-api-access-v2p8x\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.475982 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.475959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-telemeter-client-tls\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577292 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-serving-certs-ca-bundle\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577292 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2p8x\" (UniqueName: \"kubernetes.io/projected/14ddbcfc-db2b-433d-a8a0-f43adce481de-kube-api-access-v2p8x\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577575 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:50.577575 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-telemeter-client-tls\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577575 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577575 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577575 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-federate-client-tls\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577575 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-secret-telemeter-client\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.577873 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-metrics-client-ca\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.578194 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.577975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-serving-certs-ca-bundle\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.578311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.578278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-metrics-client-ca\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.579218 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.579188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ddbcfc-db2b-433d-a8a0-f43adce481de-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.581038 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.581010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6f2ad05c-2bd6-45f9-8c44-8b4a3b839606-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xkvrq\" (UID: \"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:50.581693 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.581670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-secret-telemeter-client\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.581850 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.581825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-federate-client-tls\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.581850 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.581843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.581963 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.581839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/14ddbcfc-db2b-433d-a8a0-f43adce481de-telemeter-client-tls\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.586139 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.586116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2p8x\" (UniqueName: \"kubernetes.io/projected/14ddbcfc-db2b-433d-a8a0-f43adce481de-kube-api-access-v2p8x\") pod \"telemeter-client-7b84dcb675-lm529\" (UID: \"14ddbcfc-db2b-433d-a8a0-f43adce481de\") " pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.688787 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.688710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529"
Jul 23 03:56:50.804280 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:50.804241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:51.430256 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.426315    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:56:51.430793 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.430770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.433729 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.433710    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 03:56:51.434362 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.434332    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 03:56:51.434511 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.434456    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l5pmz\""
Jul 23 03:56:51.434902 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.434885    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.435440    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.435568    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.435814    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.435880    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.436043    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4qgapneqsfqg7\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.436093    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 03:56:51.436744 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.436510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 03:56:51.437144 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.437025    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 03:56:51.437509 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.437485    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 03:56:51.441018 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.437798    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 03:56:51.441018 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.439426    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 03:56:51.443172 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.443151    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 03:56:51.450311 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.450289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:56:51.487008 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.486842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-config\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487008 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.486915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487008 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.486959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6889j\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-kube-api-access-6889j\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.487950 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.487862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.491091 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.488919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6889j\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-kube-api-access-6889j\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-config\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.601661 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.602609 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.602609 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.602609 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.595704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.602609 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.598396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.602609 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.598895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.611529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.603911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.611529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.605317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.611529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.606809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.611529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.608679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.611529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.609837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.611529 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.609952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.613691 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.613633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.613884 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.613858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.616382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.616354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.617680 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.617639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-config\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.623607 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.623580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6889j\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-kube-api-access-6889j\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.625126 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.625011    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.625599 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.625495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.626737 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.626251    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.626737 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.626296    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.626893 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.626752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.627723 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.627484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.630562 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.630479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:51.681021 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.680744    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-744786bd49-t9xps"]
Jul 23 03:56:51.720949 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.720869    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"]
Jul 23 03:56:51.751666 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.751567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-867b956d86-md2s6"]
Jul 23 03:56:51.751666 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.751624    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b84dcb675-lm529"]
Jul 23 03:56:51.813583 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:51.813404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:56:52.001997 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.001957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:56:52.003379 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:56:52.003356    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d65958b_9950_4afd_aad7_c2dd73596f75.slice/crio-a061b5fc932dbce2879f2d96ecc500a0f579f997ea6aee90cf4d0a3244dbcbef WatchSource:0}: Error finding container a061b5fc932dbce2879f2d96ecc500a0f579f997ea6aee90cf4d0a3244dbcbef: Status 404 returned error can't find the container with id a061b5fc932dbce2879f2d96ecc500a0f579f997ea6aee90cf4d0a3244dbcbef
Jul 23 03:56:52.266678 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.266643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9k5f" event={"ID":"bd539a0e-afe7-4f62-82e1-e334bfb6dd83","Type":"ContainerStarted","Data":"890f0ac543a2fdcfba6d3c514a6cf2ab3ba4d7e92e95695471dd5480c32e284e"}
Jul 23 03:56:52.266854 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.266735    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:56:52.268626 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.268597    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2" exitCode=0
Jul 23 03:56:52.268729 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.268627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2"}
Jul 23 03:56:52.271399 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.271377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk" event={"ID":"c4da9a14-ffa8-4545-8826-343b63133c1f","Type":"ContainerStarted","Data":"0791063d832ea6066e2b623c24193f2b20cf3145de8abf42b92cf0fa7bae208b"}
Jul 23 03:56:52.271525 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.271449    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk" event={"ID":"c4da9a14-ffa8-4545-8826-343b63133c1f","Type":"ContainerStarted","Data":"fe713331b41d60e3c4d15cd5c821c7a393cf035a7723cf21af8130b5c0d8733e"}
Jul 23 03:56:52.271525 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.271470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk" event={"ID":"c4da9a14-ffa8-4545-8826-343b63133c1f","Type":"ContainerStarted","Data":"2b73b0df2c957585dc3688a2cb8e0935409dca8c9b91b59665115e4bd4a4fce1"}
Jul 23 03:56:52.273313 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.273290    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da" exitCode=0
Jul 23 03:56:52.273438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.273374    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"}
Jul 23 03:56:52.273510 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.273490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"a061b5fc932dbce2879f2d96ecc500a0f579f997ea6aee90cf4d0a3244dbcbef"}
Jul 23 03:56:52.274606 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.274442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq" event={"ID":"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606","Type":"ContainerStarted","Data":"48380aae49892caae193d85c296ca4a0335b7e3727a76e56424881147a8ece54"}
Jul 23 03:56:52.276002 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.275977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529" event={"ID":"14ddbcfc-db2b-433d-a8a0-f43adce481de","Type":"ContainerStarted","Data":"1de7619e4cfd5157ba416576e4372e5134253fe7e92fb55950aca3dc36739ca9"}
Jul 23 03:56:52.277638 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.277615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9cwq" event={"ID":"4052be7b-0b50-40e8-a250-3e251a80d7d9","Type":"ContainerStarted","Data":"270b6a0e35cc745043d9ed1538bd2de510c3183e6ac73aad8dd538dbb4bbb1e0"}
Jul 23 03:56:52.279171 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.279150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-867b956d86-md2s6" event={"ID":"44408668-1169-4463-a5f3-2721bf8da747","Type":"ContainerStarted","Data":"7039fca3030bc61010cce78d57c9f4f99fe2a3e27f291144efd5da8efc58f578"}
Jul 23 03:56:52.280383 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.280357    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"4bf9f19d1c5f4bf9440b683d1242d70cec1038fc21ea823da43f1b8c984a7370"}
Jul 23 03:56:52.285948 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.285899    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q9k5f" podStartSLOduration=67.0059361 podStartE2EDuration="1m13.285881096s" totalImagesPullingTime="6.279944996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:52.284336807 +0000 UTC m=+73.003868430" watchObservedRunningTime="2026-07-23 03:56:52.285881096 +0000 UTC m=+73.005412709"
Jul 23 03:56:52.308806 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.308742    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-x6fqk" podStartSLOduration=2.802614233 podStartE2EDuration="7.308723728s" totalImagesPullingTime="4.506109495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:52.308133196 +0000 UTC m=+73.027664807" watchObservedRunningTime="2026-07-23 03:56:52.308723728 +0000 UTC m=+73.028255337"
Jul 23 03:56:52.361523 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:52.361473    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z9cwq" podStartSLOduration=66.84262379 podStartE2EDuration="1m13.361456337s" totalImagesPullingTime="6.518832547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:52.328401412 +0000 UTC m=+73.047933023" watchObservedRunningTime="2026-07-23 03:56:52.361456337 +0000 UTC m=+73.080987925"
Jul 23 03:56:57.308135 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.307863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"}
Jul 23 03:56:57.308135 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.307983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"}
Jul 23 03:56:57.308135 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.308024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"}
Jul 23 03:56:57.308135 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.308062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"}
Jul 23 03:56:57.312153 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.311338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq" event={"ID":"6f2ad05c-2bd6-45f9-8c44-8b4a3b839606","Type":"ContainerStarted","Data":"836065af85025b379b69dccf42f56175caddfa8112da8e9c248d3b98521f8c72"}
Jul 23 03:56:57.312153 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.312112    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:57.315707 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.315020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529" event={"ID":"14ddbcfc-db2b-433d-a8a0-f43adce481de","Type":"ContainerStarted","Data":"581d39e5de5a515413f566364207bae00a788ec2d87aa5acad31cf5f85ea8d4c"}
Jul 23 03:56:57.315707 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.315071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529" event={"ID":"14ddbcfc-db2b-433d-a8a0-f43adce481de","Type":"ContainerStarted","Data":"9d230bc6e7953d42dd77d4125369e1bfca86ce4a59d5b55b8aee1516d518869e"}
Jul 23 03:56:57.315707 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.315100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529" event={"ID":"14ddbcfc-db2b-433d-a8a0-f43adce481de","Type":"ContainerStarted","Data":"b83492e4bb35d6947f733c8f731967f6d768441c78d70fd48290903b35fcc8b7"}
Jul 23 03:56:57.319046 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.319024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-867b956d86-md2s6" event={"ID":"44408668-1169-4463-a5f3-2721bf8da747","Type":"ContainerStarted","Data":"815061bb52baeb19d769993e48a1f9408b48ff98e5e6fa1eebf26832805b8a9e"}
Jul 23 03:56:57.320057 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.319828    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq"
Jul 23 03:56:57.323242 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.323206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"3fbd73063e909d4c190b3cbf55cbc08962283eb6b3d73d01808928fb22a70e78"}
Jul 23 03:56:57.323507 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.323483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"417daa8532e216d79d2c1347c640f2b3aeb5973fbd32fab0aa98a5b418eef56a"}
Jul 23 03:56:57.323659 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.323646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"7e29f5de1146fe6d51d944bfcc7c394bbf9bee1f04ed9988d0cf53b016300599"}
Jul 23 03:56:57.330762 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.329458    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xkvrq" podStartSLOduration=3.745541564 podStartE2EDuration="8.329430715s" totalImagesPullingTime="4.583889151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:57.327092327 +0000 UTC m=+78.046623938" watchObservedRunningTime="2026-07-23 03:56:57.329430715 +0000 UTC m=+78.048962321"
Jul 23 03:56:57.330762 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.330713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee"}
Jul 23 03:56:57.330994 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.330792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a"}
Jul 23 03:56:57.330994 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.330835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee"}
Jul 23 03:56:57.330994 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.330874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58"}
Jul 23 03:56:57.365093 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.365044    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-867b956d86-md2s6" podStartSLOduration=3.803574116 podStartE2EDuration="8.365024295s" totalImagesPullingTime="4.561450179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:57.347933588 +0000 UTC m=+78.067465198" watchObservedRunningTime="2026-07-23 03:56:57.365024295 +0000 UTC m=+78.084555884"
Jul 23 03:56:57.385964 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:57.385920    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b84dcb675-lm529" podStartSLOduration=2.833102101 podStartE2EDuration="7.385903834s" totalImagesPullingTime="4.552801733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:57.382981325 +0000 UTC m=+78.102512935" watchObservedRunningTime="2026-07-23 03:56:57.385903834 +0000 UTC m=+78.105435422"
Jul 23 03:56:58.338531 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.338442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"}
Jul 23 03:56:58.338531 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.338517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerStarted","Data":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"}
Jul 23 03:56:58.341371 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.341342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"89fcf13c86d573be0cfd943f3370565e018435f75ac02cd891f23ba33cb1d1b3"}
Jul 23 03:56:58.341487 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.341402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"4536faf302b6933003ab662cb211a64e1bb481755da2b117e8798e54702926f6"}
Jul 23 03:56:58.341487 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.341458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" event={"ID":"4bc3eb25-192a-4840-89b8-912db55cb57b","Type":"ContainerStarted","Data":"0663dc8b2ecf39a2a9a49814d98e3649ffdc096b1536d60b41ac1d1e6c821ede"}
Jul 23 03:56:58.341615 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.341488    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:56:58.344846 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.344819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4"}
Jul 23 03:56:58.344956 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.344876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerStarted","Data":"07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2"}
Jul 23 03:56:58.369675 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.369270    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.835103272 podStartE2EDuration="7.369250005s" totalImagesPullingTime="4.513835501s" totalInitContainerRuntime="20.311232ms" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:58.368123604 +0000 UTC m=+79.087655215" watchObservedRunningTime="2026-07-23 03:56:58.369250005 +0000 UTC m=+79.088781616"
Jul 23 03:56:58.396309 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.396257    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.740269046 podStartE2EDuration="12.396241079s" totalImagesPullingTime="9.631646856s" totalInitContainerRuntime="24.325177ms" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:58.39442745 +0000 UTC m=+79.113959059" watchObservedRunningTime="2026-07-23 03:56:58.396241079 +0000 UTC m=+79.115772689"
Jul 23 03:56:58.427819 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:56:58.427094    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps" podStartSLOduration=5.868924007 podStartE2EDuration="11.427072089s" totalImagesPullingTime="5.558148082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:58.425571759 +0000 UTC m=+79.145103394" watchObservedRunningTime="2026-07-23 03:56:58.427072089 +0000 UTC m=+79.146603701"
Jul 23 03:57:01.814154 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:01.814004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:57:04.355425 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:04.355393    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-744786bd49-t9xps"
Jul 23 03:57:09.791824 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:09.791791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:57:09.792266 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:09.791878    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:57:23.288873 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:23.288844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q9k5f"
Jul 23 03:57:29.797570 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:29.797535    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:57:29.801369 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:29.801341    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-867b956d86-md2s6"
Jul 23 03:57:51.813833 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:51.813798    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:57:51.832903 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:51.832878    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:57:52.581804 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:57:52.581779    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:05.743208 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743173    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:58:05.743701 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743497    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="alertmanager" containerID="cri-o://774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58" gracePeriod=120
Jul 23 03:58:05.743701 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743562    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-metric" containerID="cri-o://07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2" gracePeriod=120
Jul 23 03:58:05.743701 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743630    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="config-reloader" containerID="cri-o://bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee" gracePeriod=120
Jul 23 03:58:05.743875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743680    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="prom-label-proxy" containerID="cri-o://f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4" gracePeriod=120
Jul 23 03:58:05.743875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743560    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-web" containerID="cri-o://a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a" gracePeriod=120
Jul 23 03:58:05.743875 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:05.743592    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy" containerID="cri-o://5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee" gracePeriod=120
Jul 23 03:58:06.625851 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625812    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4" exitCode=0
Jul 23 03:58:06.625851 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625841    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2" exitCode=0
Jul 23 03:58:06.625851 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625852    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee" exitCode=0
Jul 23 03:58:06.625851 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625860    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee" exitCode=0
Jul 23 03:58:06.626119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625871    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58" exitCode=0
Jul 23 03:58:06.626119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4"}
Jul 23 03:58:06.626119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.625976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2"}
Jul 23 03:58:06.626119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.626001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee"}
Jul 23 03:58:06.626119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.626027    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee"}
Jul 23 03:58:06.626119 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.626049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58"}
Jul 23 03:58:06.987472 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:06.987449    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.070031 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070000    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-main-db\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070204 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070204 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070115    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-config-out\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070204 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trxsk\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-kube-api-access-trxsk\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070204 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-web\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070431 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-metrics-client-ca\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070431 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070286    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-cluster-tls-config\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070431 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070327    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-web-config\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070431 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 03:58:07.070431 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-config-volume\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070431 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070390    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-trusted-ca-bundle\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070704 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070459    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070704 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070489    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-tls-assets\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070704 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070523    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-main-tls\") pod \"376ec373-6628-4da1-940a-fd212a8432be\" (UID: \"376ec373-6628-4da1-940a-fd212a8432be\") "
Jul 23 03:58:07.070838 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.070816    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-main-db\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.071095 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.071059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:07.071585 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.071514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:07.073388 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.073343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.073539 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.073454    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.073539 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.073498    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-kube-api-access-trxsk" (OuterVolumeSpecName: "kube-api-access-trxsk") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "kube-api-access-trxsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:58:07.073746 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.073716    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-config-out" (OuterVolumeSpecName: "config-out") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 03:58:07.074028 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.073945    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.074243 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.074224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-config-volume" (OuterVolumeSpecName: "config-volume") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.075515 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.075493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:58:07.075785 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.075771    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.078728 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.078708    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.085946 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.085924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-web-config" (OuterVolumeSpecName: "web-config") pod "376ec373-6628-4da1-940a-fd212a8432be" (UID: "376ec373-6628-4da1-940a-fd212a8432be"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171347    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-cluster-tls-config\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171375    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-web-config\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171385    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-config-volume\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171394    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171405    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171436    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-tls-assets\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171438 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171444    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-main-tls\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171730 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171453    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171730 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171462    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/376ec373-6628-4da1-940a-fd212a8432be-config-out\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171730 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171470    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trxsk\" (UniqueName: \"kubernetes.io/projected/376ec373-6628-4da1-940a-fd212a8432be-kube-api-access-trxsk\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171730 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171478    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376ec373-6628-4da1-940a-fd212a8432be-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.171730 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.171487    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376ec373-6628-4da1-940a-fd212a8432be-metrics-client-ca\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:07.632868 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.632836    2563 generic.go:356] "Generic (PLEG): container finished" podID="376ec373-6628-4da1-940a-fd212a8432be" containerID="a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a" exitCode=0
Jul 23 03:58:07.633028 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.632915    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.633028 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.632914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a"}
Jul 23 03:58:07.633028 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.632988    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"376ec373-6628-4da1-940a-fd212a8432be","Type":"ContainerDied","Data":"7a7acd3ff0a1e4efc2811896f4ebc1f0937420bc47a774d0340f50be44a4b56e"}
Jul 23 03:58:07.633137 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.633033    2563 scope.go:117] "RemoveContainer" containerID="f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4"
Jul 23 03:58:07.641440 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.641403    2563 scope.go:117] "RemoveContainer" containerID="07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2"
Jul 23 03:58:07.648296 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.648269    2563 scope.go:117] "RemoveContainer" containerID="5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee"
Jul 23 03:58:07.655481 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.655463    2563 scope.go:117] "RemoveContainer" containerID="a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a"
Jul 23 03:58:07.657152 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.657132    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:58:07.662100 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.662076    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:58:07.663712 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.663697    2563 scope.go:117] "RemoveContainer" containerID="bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee"
Jul 23 03:58:07.670770 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.670752    2563 scope.go:117] "RemoveContainer" containerID="774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58"
Jul 23 03:58:07.677867 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.677850    2563 scope.go:117] "RemoveContainer" containerID="526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2"
Jul 23 03:58:07.686793 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.686773    2563 scope.go:117] "RemoveContainer" containerID="f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4"
Jul 23 03:58:07.687160 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.687139    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4\": container with ID starting with f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4 not found: ID does not exist" containerID="f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4"
Jul 23 03:58:07.687227 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.687170    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4"} err="failed to get container status \"f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4\": rpc error: code = NotFound desc = could not find container \"f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4\": container with ID starting with f8a19a91ee5db969209fa9782d277e8f1129fb74cbb0f23f79ae45363aae22c4 not found: ID does not exist"
Jul 23 03:58:07.687227 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.687205    2563 scope.go:117] "RemoveContainer" containerID="07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2"
Jul 23 03:58:07.687541 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.687516    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2\": container with ID starting with 07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2 not found: ID does not exist" containerID="07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2"
Jul 23 03:58:07.687625 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.687546    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2"} err="failed to get container status \"07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2\": rpc error: code = NotFound desc = could not find container \"07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2\": container with ID starting with 07c5bcc95740b7c3b0f0249536d03562f43dd0ec90347b0cc089ccf40d6808a2 not found: ID does not exist"
Jul 23 03:58:07.687625 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.687564    2563 scope.go:117] "RemoveContainer" containerID="5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee"
Jul 23 03:58:07.687818 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.687801    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee\": container with ID starting with 5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee not found: ID does not exist" containerID="5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee"
Jul 23 03:58:07.687859 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.687825    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee"} err="failed to get container status \"5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee\": rpc error: code = NotFound desc = could not find container \"5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee\": container with ID starting with 5377d5b4f2238af603030658a792dc660c3d874bd0555c513f02aebcc44881ee not found: ID does not exist"
Jul 23 03:58:07.687859 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.687841    2563 scope.go:117] "RemoveContainer" containerID="a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a"
Jul 23 03:58:07.688076 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.688058    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a\": container with ID starting with a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a not found: ID does not exist" containerID="a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a"
Jul 23 03:58:07.688127 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688082    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a"} err="failed to get container status \"a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a\": rpc error: code = NotFound desc = could not find container \"a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a\": container with ID starting with a8869f8f19a3f1721c83e487950eb387f65bdfe65f5dac9eb0dbc0bd578a372a not found: ID does not exist"
Jul 23 03:58:07.688127 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688097    2563 scope.go:117] "RemoveContainer" containerID="bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee"
Jul 23 03:58:07.688310 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.688295    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee\": container with ID starting with bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee not found: ID does not exist" containerID="bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee"
Jul 23 03:58:07.688350 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688313    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee"} err="failed to get container status \"bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee\": rpc error: code = NotFound desc = could not find container \"bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee\": container with ID starting with bca04aac3fa5ef3f762335e40f58134bc3fef993d07a52904265484d9e2120ee not found: ID does not exist"
Jul 23 03:58:07.688350 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688324    2563 scope.go:117] "RemoveContainer" containerID="774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58"
Jul 23 03:58:07.688606 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.688586    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58\": container with ID starting with 774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58 not found: ID does not exist" containerID="774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58"
Jul 23 03:58:07.688654 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688611    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58"} err="failed to get container status \"774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58\": rpc error: code = NotFound desc = could not find container \"774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58\": container with ID starting with 774e22b5c505f0031e55a3de570fc0473b259e8b698c996ed53c5e3830895f58 not found: ID does not exist"
Jul 23 03:58:07.688654 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688626    2563 scope.go:117] "RemoveContainer" containerID="526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2"
Jul 23 03:58:07.688850 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:07.688834    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2\": container with ID starting with 526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2 not found: ID does not exist" containerID="526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2"
Jul 23 03:58:07.688902 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.688854    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2"} err="failed to get container status \"526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2\": rpc error: code = NotFound desc = could not find container \"526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2\": container with ID starting with 526f2f618783e5eeba441d7ba718012dbf8afd68f1e9a842a01edc4ef802eaf2 not found: ID does not exist"
Jul 23 03:58:07.691264 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:58:07.691567 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691553    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-metric"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691569    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-metric"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691579    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="prom-label-proxy"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691584    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="prom-label-proxy"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691595    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="config-reloader"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691601    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="config-reloader"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691611    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy"
Jul 23 03:58:07.691619 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691616    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691625    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="init-config-reloader"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691630    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="init-config-reloader"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691636    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="alertmanager"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691641    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="alertmanager"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691650    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-web"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691655    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-web"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691698    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="config-reloader"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691705    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="prom-label-proxy"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691713    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-metric"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691721    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy-web"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691727    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="alertmanager"
Jul 23 03:58:07.691809 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.691734    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="376ec373-6628-4da1-940a-fd212a8432be" containerName="kube-rbac-proxy"
Jul 23 03:58:07.698692 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.697014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.699400 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.699381    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bnjnp\""
Jul 23 03:58:07.699534 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.699430    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 03:58:07.699605 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.699567    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 03:58:07.699757 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.699741    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 03:58:07.699850 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.699812    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 03:58:07.699907 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.699883    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 03:58:07.700456 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.700438    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 03:58:07.700733 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.700715    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 03:58:07.700909 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.700751    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 03:58:07.706347 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.706319    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 03:58:07.712388 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.712364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:58:07.749860 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.749832    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="376ec373-6628-4da1-940a-fd212a8432be" path="/var/lib/kubelet/pods/376ec373-6628-4da1-940a-fd212a8432be/volumes"
Jul 23 03:58:07.775703 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.775833 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-config-out\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.775833 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.775833 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775797    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-web-config\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.775978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.776007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms5kc\" (UniqueName: \"kubernetes.io/projected/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-kube-api-access-ms5kc\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776193 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.776045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776193 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.776081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776193 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.776098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.776193 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.776118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-config-volume\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877182 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-config-volume\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-config-out\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-web-config\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms5kc\" (UniqueName: \"kubernetes.io/projected/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-kube-api-access-ms5kc\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.877641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.877523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.878505 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.878480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.879704 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.878832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.879704 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.879615    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.880421 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.880380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-config-volume\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.880502 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.880432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.880994 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.880812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.880994 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.880944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-config-out\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.881273 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.881238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.881608 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.881586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.882078 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.882055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.882078 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.882074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-web-config\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.882489 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.882473    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:07.887785 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:07.887738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms5kc\" (UniqueName: \"kubernetes.io/projected/bcb81eff-4cca-4e3d-88ca-53608d2cf5ce-kube-api-access-ms5kc\") pod \"alertmanager-main-0\" (UID: \"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:08.009209 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:08.009172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 03:58:08.158377 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:58:08.158344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb81eff_4cca_4e3d_88ca_53608d2cf5ce.slice/crio-5ec415bce61ec1beecd6752da5cad13a2b1b8ef8e74d796d4eb0c1a142731948 WatchSource:0}: Error finding container 5ec415bce61ec1beecd6752da5cad13a2b1b8ef8e74d796d4eb0c1a142731948: Status 404 returned error can't find the container with id 5ec415bce61ec1beecd6752da5cad13a2b1b8ef8e74d796d4eb0c1a142731948
Jul 23 03:58:08.159889 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:08.159866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 03:58:08.639225 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:08.639185    2563 generic.go:356] "Generic (PLEG): container finished" podID="bcb81eff-4cca-4e3d-88ca-53608d2cf5ce" containerID="11c13373047fdf88c8fc0554d50f1eed741856df22e3171450b2253799a58767" exitCode=0
Jul 23 03:58:08.639365 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:08.639281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerDied","Data":"11c13373047fdf88c8fc0554d50f1eed741856df22e3171450b2253799a58767"}
Jul 23 03:58:08.639365 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:08.639351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"5ec415bce61ec1beecd6752da5cad13a2b1b8ef8e74d796d4eb0c1a142731948"}
Jul 23 03:58:09.646527 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.646491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"f6531f0185a6416100435d171c6d334521113760eed5a73d6f1b48036fab681b"}
Jul 23 03:58:09.646933 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.646584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"1188756ba2f9341bf17fd5c3154b06a6b857f27520fddb777685df6f83903d56"}
Jul 23 03:58:09.646933 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.646611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"243d8a7f0a571f35200070ab8cc939e6b6f87872cf33e73dbdbd7963060013d2"}
Jul 23 03:58:09.646933 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.646639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"2429f12618076c716a9bbfafbebf9e399d9e3669021857a5462df2f9211a8335"}
Jul 23 03:58:09.646933 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.646663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"6a5ac99fb3ded262cc018c481a9b94ae1560b74c553787e6aa34cf9e02d379c7"}
Jul 23 03:58:09.646933 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.646684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcb81eff-4cca-4e3d-88ca-53608d2cf5ce","Type":"ContainerStarted","Data":"6e15ad43ea97983026eb7dec6491d8e2e008395f60e661c80f4986ba46bb7791"}
Jul 23 03:58:09.679072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.679019    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.663591084 podStartE2EDuration="2.679003763s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.412679ms" isStatefulPod=false podCreationTimestamp="2026-07-23 03:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:58:09.67722622 +0000 UTC m=+150.396757830" watchObservedRunningTime="2026-07-23 03:58:09.679003763 +0000 UTC m=+150.398535373"
Jul 23 03:58:09.985232 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.985193    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:58:09.985590 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.985558    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="prometheus" containerID="cri-o://6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33" gracePeriod=600
Jul 23 03:58:09.985802 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.985768    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-thanos" containerID="cri-o://071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d" gracePeriod=600
Jul 23 03:58:09.985930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.985863    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="config-reloader" containerID="cri-o://088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8" gracePeriod=600
Jul 23 03:58:09.985930 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.985867    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy" containerID="cri-o://cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a" gracePeriod=600
Jul 23 03:58:09.986035 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.986015    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-web" containerID="cri-o://0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac" gracePeriod=600
Jul 23 03:58:09.986233 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:09.986212    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="thanos-sidecar" containerID="cri-o://7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca" gracePeriod=600
Jul 23 03:58:10.246151 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.246088    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.309969 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.309935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-trusted-ca-bundle\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310000    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-serving-certs-ca-bundle\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-0\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310059    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-kubelet-serving-ca-bundle\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310091    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-metrics-client-certs\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310128 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-db\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310155    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-tls\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310180    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6889j\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-kube-api-access-6889j\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310204    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-thanos-prometheus-http-client-file\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-tls-assets\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310264    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-config-out\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310293    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-1\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310371    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-metrics-client-ca\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-grpc-tls\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310456    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-kube-rbac-proxy\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310523    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-config\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310592    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-web-config\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.310798 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310620    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-2\") pod \"5d65958b-9950-4afd-aad7-c2dd73596f75\" (UID: \"5d65958b-9950-4afd-aad7-c2dd73596f75\") "
Jul 23 03:58:10.311117 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.310880    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.311117 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.311006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.311243 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.311221    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.311713 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.311460    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.311713 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.311507    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.312877 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.312545    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.314141 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.314112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:10.314243 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.314221    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 03:58:10.315216 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.315140    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:58:10.317469 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.317351    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.317901 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.317628    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-config" (OuterVolumeSpecName: "config") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.317901 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.317758    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.317901 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.317879    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.318314 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.318283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.318498 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.318480    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.319234 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.319188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.319319 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.319229    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-kube-api-access-6889j" (OuterVolumeSpecName: "kube-api-access-6889j") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "kube-api-access-6889j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:58:10.319319 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.319274    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-config-out" (OuterVolumeSpecName: "config-out") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 03:58:10.319319 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.319299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.331161 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.330773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-web-config" (OuterVolumeSpecName: "web-config") pod "5d65958b-9950-4afd-aad7-c2dd73596f75" (UID: "5d65958b-9950-4afd-aad7-c2dd73596f75"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:10.412259 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412219    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412259 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412251    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412259 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412262    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412272    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-metrics-client-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412281    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-db\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412290    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412298    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6889j\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-kube-api-access-6889j\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412306    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412315    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d65958b-9950-4afd-aad7-c2dd73596f75-tls-assets\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412323    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d65958b-9950-4afd-aad7-c2dd73596f75-config-out\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412330    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412339    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412346    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-configmap-metrics-client-ca\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412354    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-grpc-tls\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412362    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-kube-rbac-proxy\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412370    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412379    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-config\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412387    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d65958b-9950-4afd-aad7-c2dd73596f75-web-config\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.412512 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.412396    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d65958b-9950-4afd-aad7-c2dd73596f75-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653344    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d" exitCode=0
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653373    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a" exitCode=0
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653381    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac" exitCode=0
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653389    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca" exitCode=0
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653397    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8" exitCode=0
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653404    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33" exitCode=0
Jul 23 03:58:10.653450 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"}
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653465    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"}
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"}
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"}
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"}
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653605    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653606    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"}
Jul 23 03:58:10.654041 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.653745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d65958b-9950-4afd-aad7-c2dd73596f75","Type":"ContainerDied","Data":"a061b5fc932dbce2879f2d96ecc500a0f579f997ea6aee90cf4d0a3244dbcbef"}
Jul 23 03:58:10.667336 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.667317    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.674780 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.674765    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.682699 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.682678    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.683221 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.683198    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:58:10.686518 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.686499    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:58:10.690156 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.690138    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.696840 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.696825    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.707802 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.707778    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.714935 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.714910    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:58:10.715703 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715676    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="config-reloader"
Jul 23 03:58:10.715703 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715699    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="config-reloader"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715733    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715743    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715755    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="prometheus"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715763    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="prometheus"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715775    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-web"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715783    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-web"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715805    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="thanos-sidecar"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715814    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="thanos-sidecar"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715824    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-thanos"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715832    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-thanos"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715846    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="init-config-reloader"
Jul 23 03:58:10.715879 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715853    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="init-config-reloader"
Jul 23 03:58:10.716382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715927    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-web"
Jul 23 03:58:10.716382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715940    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy"
Jul 23 03:58:10.716382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715949    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="kube-rbac-proxy-thanos"
Jul 23 03:58:10.716382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715960    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="thanos-sidecar"
Jul 23 03:58:10.716382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715969    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="config-reloader"
Jul 23 03:58:10.716382 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.715978    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" containerName="prometheus"
Jul 23 03:58:10.716717 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.716501    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.716899 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.716869    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.717076 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.716909    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"} err="failed to get container status \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist"
Jul 23 03:58:10.717076 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.716936    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.717257 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.717240    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.717296 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.717265    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"} err="failed to get container status \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist"
Jul 23 03:58:10.717296 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.717281    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.717596 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.717575    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.717631 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.717603    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"} err="failed to get container status \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist"
Jul 23 03:58:10.717631 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.717620    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.717855 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.717841    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.717901 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.717859    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"} err="failed to get container status \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist"
Jul 23 03:58:10.717901 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.717872    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.718091 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.718072    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.718148 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718100    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"} err="failed to get container status \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist"
Jul 23 03:58:10.718148 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718121    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.718340 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.718325    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.718374 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718344    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"} err="failed to get container status \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist"
Jul 23 03:58:10.718374 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718355    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.718569 ip-10-0-131-0 kubenswrapper[2563]: E0723 03:58:10.718555    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.718620 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718572    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"} err="failed to get container status \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist"
Jul 23 03:58:10.718620 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718584    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.718795 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718777    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"} err="failed to get container status \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist"
Jul 23 03:58:10.718856 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718797    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.718996 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718978    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"} err="failed to get container status \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist"
Jul 23 03:58:10.719056 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.718997    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.719181 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719166    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"} err="failed to get container status \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist"
Jul 23 03:58:10.719241 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719183    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.719402 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719374    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"} err="failed to get container status \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist"
Jul 23 03:58:10.719554 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719404    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.719693 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719673    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"} err="failed to get container status \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist"
Jul 23 03:58:10.719737 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719693    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.719911 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719894    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"} err="failed to get container status \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist"
Jul 23 03:58:10.719969 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.719912    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.720227 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.720208    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"} err="failed to get container status \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist"
Jul 23 03:58:10.720286 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.720229    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.720523 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.720496    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"} err="failed to get container status \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist"
Jul 23 03:58:10.720523 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.720522    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.720824 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.720797    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"} err="failed to get container status \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist"
Jul 23 03:58:10.720905 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.720827    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.722960 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.721726    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"} err="failed to get container status \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist"
Jul 23 03:58:10.722960 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.721773    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.722960 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.721838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.722960 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.722056    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"} err="failed to get container status \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist"
Jul 23 03:58:10.722960 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.722098    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.722960 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.722950    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"} err="failed to get container status \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist"
Jul 23 03:58:10.723310 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.722974    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.723358 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.723334    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"} err="failed to get container status \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist"
Jul 23 03:58:10.723433 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.723357    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.723655 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.723638    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"} err="failed to get container status \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist"
Jul 23 03:58:10.723714 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.723656    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.723885 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.723862    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"} err="failed to get container status \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist"
Jul 23 03:58:10.723885 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.723885    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.724114 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724080    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"} err="failed to get container status \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist"
Jul 23 03:58:10.724114 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724106    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.724265 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724252    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 03:58:10.724354 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724322    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"} err="failed to get container status \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist"
Jul 23 03:58:10.724465 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724354    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.724637 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724603    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"} err="failed to get container status \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist"
Jul 23 03:58:10.724637 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724631    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.724900 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724872    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"} err="failed to get container status \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist"
Jul 23 03:58:10.724900 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724897    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.725055 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.724991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 03:58:10.725055 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l5pmz\""
Jul 23 03:58:10.725205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725021    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4qgapneqsfqg7\""
Jul 23 03:58:10.725205 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725141    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 03:58:10.725328 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725195    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"} err="failed to get container status \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist"
Jul 23 03:58:10.725328 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725225    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.725328 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725242    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 03:58:10.725328 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725263    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 03:58:10.725328 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725266    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 03:58:10.725651 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725458    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"} err="failed to get container status \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist"
Jul 23 03:58:10.725651 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725477    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.725651 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 03:58:10.725651 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 03:58:10.725884 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725660    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 03:58:10.725990 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725974    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 03:58:10.726072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.725967    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"} err="failed to get container status \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist"
Jul 23 03:58:10.726072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726011    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.726072 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726062    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 03:58:10.726344 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726089    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 03:58:10.726344 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726306    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"} err="failed to get container status \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist"
Jul 23 03:58:10.726344 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726329    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.726616 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726579    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"} err="failed to get container status \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist"
Jul 23 03:58:10.726616 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726601    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.726874 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726847    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"} err="failed to get container status \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist"
Jul 23 03:58:10.726924 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.726877    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.727169 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727139    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"} err="failed to get container status \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist"
Jul 23 03:58:10.727224 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727168    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.727482 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727388    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"} err="failed to get container status \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist"
Jul 23 03:58:10.727482 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727445    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.727679 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727661    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"} err="failed to get container status \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist"
Jul 23 03:58:10.727721 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727680    2563 scope.go:117] "RemoveContainer" containerID="071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"
Jul 23 03:58:10.727895 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727873    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d"} err="failed to get container status \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": rpc error: code = NotFound desc = could not find container \"071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d\": container with ID starting with 071828c1c4671618e0b7f982191ae39fdd211ca5a5ca16412be2608463714e4d not found: ID does not exist"
Jul 23 03:58:10.727990 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.727899    2563 scope.go:117] "RemoveContainer" containerID="cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"
Jul 23 03:58:10.728127 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.728100    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a"} err="failed to get container status \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": rpc error: code = NotFound desc = could not find container \"cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a\": container with ID starting with cd72f69e6e4bfb2aebaa10f23ff4f98855a342ca871f5b3eb86064da12377b8a not found: ID does not exist"
Jul 23 03:58:10.728234 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.728128    2563 scope.go:117] "RemoveContainer" containerID="0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"
Jul 23 03:58:10.728469 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.728380    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac"} err="failed to get container status \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": rpc error: code = NotFound desc = could not find container \"0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac\": container with ID starting with 0269acdf4054eb7f8c37be1fcbb82a66e35bf24ceea1a48087780c3f9b9784ac not found: ID does not exist"
Jul 23 03:58:10.728469 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.728425    2563 scope.go:117] "RemoveContainer" containerID="7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"
Jul 23 03:58:10.728959 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.728933    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca"} err="failed to get container status \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": rpc error: code = NotFound desc = could not find container \"7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca\": container with ID starting with 7daca407837c97a08dc749f4b1173f94a60c6baf355044a92e134fde2902cdca not found: ID does not exist"
Jul 23 03:58:10.729052 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.728959    2563 scope.go:117] "RemoveContainer" containerID="088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"
Jul 23 03:58:10.729143 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.729094    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 03:58:10.729281 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.729232    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8"} err="failed to get container status \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": rpc error: code = NotFound desc = could not find container \"088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8\": container with ID starting with 088d81b634df8a684a445614bafb54c330494d930f642ee71d9738893fb7aaf8 not found: ID does not exist"
Jul 23 03:58:10.729281 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.729259    2563 scope.go:117] "RemoveContainer" containerID="6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"
Jul 23 03:58:10.729533 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.729515    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33"} err="failed to get container status \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": rpc error: code = NotFound desc = could not find container \"6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33\": container with ID starting with 6ed361089dd6f84f0b34e7c887b981b0a31c016b96bf428a4e0bc7be142b0a33 not found: ID does not exist"
Jul 23 03:58:10.729533 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.729534    2563 scope.go:117] "RemoveContainer" containerID="52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"
Jul 23 03:58:10.729843 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.729732    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da"} err="failed to get container status \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": rpc error: code = NotFound desc = could not find container \"52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da\": container with ID starting with 52d527ef5827830213ab72e6361e9ef8c18a4c5ad6b17534bfcc1fb559e1f7da not found: ID does not exist"
Jul 23 03:58:10.732365 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.732344    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 03:58:10.739318 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.739300    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:58:10.816074 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816074 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c245edbf-6e31-4c49-9523-01d215b62539-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrsbw\" (UniqueName: \"kubernetes.io/projected/c245edbf-6e31-4c49-9523-01d215b62539-kube-api-access-jrsbw\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816322 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c245edbf-6e31-4c49-9523-01d215b62539-config-out\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816356    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-config\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-web-config\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816589 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816854 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816854 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816854 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816854 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.816854 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.816690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918003 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.917922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c245edbf-6e31-4c49-9523-01d215b62539-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918003 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.917957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918003 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.917977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918003 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.917994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrsbw\" (UniqueName: \"kubernetes.io/projected/c245edbf-6e31-4c49-9523-01d215b62539-kube-api-access-jrsbw\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c245edbf-6e31-4c49-9523-01d215b62539-config-out\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-config\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-web-config\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918257 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.918652 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.919305 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.919198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.919305 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.919293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.921513 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.920027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.921513 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.921185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c245edbf-6e31-4c49-9523-01d215b62539-config-out\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.921513 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.918450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.921513 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.921441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.921513 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.921463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c245edbf-6e31-4c49-9523-01d215b62539-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.921828 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.921793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.922086 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.922059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.922175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.922067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.922175 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.922111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.922470 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.922348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-config\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.922912 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.922890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.923449 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.923396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-web-config\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.923725 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.923704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c245edbf-6e31-4c49-9523-01d215b62539-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.923899 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.923867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.923977 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.923901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.924163 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.924146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.924321 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.924300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c245edbf-6e31-4c49-9523-01d215b62539-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:10.926924 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:10.926907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrsbw\" (UniqueName: \"kubernetes.io/projected/c245edbf-6e31-4c49-9523-01d215b62539-kube-api-access-jrsbw\") pod \"prometheus-k8s-0\" (UID: \"c245edbf-6e31-4c49-9523-01d215b62539\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:11.032878 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:11.032849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:58:11.176572 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:11.176504    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 03:58:11.181630 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:58:11.181607    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc245edbf_6e31_4c49_9523_01d215b62539.slice/crio-941ec8575a443ea5d21480d86d6a4528efe72b7e6b430227466087cde6d637bb WatchSource:0}: Error finding container 941ec8575a443ea5d21480d86d6a4528efe72b7e6b430227466087cde6d637bb: Status 404 returned error can't find the container with id 941ec8575a443ea5d21480d86d6a4528efe72b7e6b430227466087cde6d637bb
Jul 23 03:58:11.659212 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:11.659179    2563 generic.go:356] "Generic (PLEG): container finished" podID="c245edbf-6e31-4c49-9523-01d215b62539" containerID="39ed93842a815ec8bd0b7d972c82f58f1cd46bb82982877cb99cd6373ac74198" exitCode=0
Jul 23 03:58:11.659664 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:11.659266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerDied","Data":"39ed93842a815ec8bd0b7d972c82f58f1cd46bb82982877cb99cd6373ac74198"}
Jul 23 03:58:11.659664 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:11.659348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"941ec8575a443ea5d21480d86d6a4528efe72b7e6b430227466087cde6d637bb"}
Jul 23 03:58:11.757865 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:11.757830    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d65958b-9950-4afd-aad7-c2dd73596f75" path="/var/lib/kubelet/pods/5d65958b-9950-4afd-aad7-c2dd73596f75/volumes"
Jul 23 03:58:12.667989 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.667956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"1e72c79ccf4a8afba5627c75b847664a7de7323f68f859f3e42acfbd16577a28"}
Jul 23 03:58:12.668430 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.668019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"2ab86ddefcb3c0aa31b909d5d25f97b493697b99fcff7741e1b04a13b6200513"}
Jul 23 03:58:12.668430 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.668043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"0030b42ab459f558a6de22c2d2806f948b196d498bf8c811d326c4032f63bad0"}
Jul 23 03:58:12.668430 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.668074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"f4d76037683a3bf8067b8032fa25c0e94cee940e39ac9340af0e3a4865045465"}
Jul 23 03:58:12.668430 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.668109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"e0740789ee79c773df02392e6e59b0e574a65ce0db0883bba31b2e826117c326"}
Jul 23 03:58:12.668430 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.668148    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c245edbf-6e31-4c49-9523-01d215b62539","Type":"ContainerStarted","Data":"7a43a6146b4b5bd06187fb1263681d5f680fbf4bbaa99b79441bf00858b88777"}
Jul 23 03:58:12.703136 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:12.703088    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.686773459 podStartE2EDuration="2.703073447s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.299988ms" isStatefulPod=false podCreationTimestamp="2026-07-23 03:58:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:58:12.69987952 +0000 UTC m=+153.419411153" watchObservedRunningTime="2026-07-23 03:58:12.703073447 +0000 UTC m=+153.422605079"
Jul 23 03:58:16.033919 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:58:16.033879    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:59:11.033906 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:11.033873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:59:11.050106 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:11.050080    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:59:11.925657 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:11.925632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 03:59:23.537561 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.537511    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9g244"]
Jul 23 03:59:23.541958 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.541941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.553082 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.553059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 03:59:23.554323 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.554309    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 03:59:23.567724 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.567699    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9g244"]
Jul 23 03:59:23.600023 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.599995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38d10468-ff88-4cb3-a9b5-793f8df5acc5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9g244\" (UID: \"38d10468-ff88-4cb3-a9b5-793f8df5acc5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.600122 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.600042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2v6\" (UniqueName: \"kubernetes.io/projected/38d10468-ff88-4cb3-a9b5-793f8df5acc5-kube-api-access-9h2v6\") pod \"cert-manager-webhook-6486ffd997-9g244\" (UID: \"38d10468-ff88-4cb3-a9b5-793f8df5acc5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.701283 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.701261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2v6\" (UniqueName: \"kubernetes.io/projected/38d10468-ff88-4cb3-a9b5-793f8df5acc5-kube-api-access-9h2v6\") pod \"cert-manager-webhook-6486ffd997-9g244\" (UID: \"38d10468-ff88-4cb3-a9b5-793f8df5acc5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.701374 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.701326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38d10468-ff88-4cb3-a9b5-793f8df5acc5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9g244\" (UID: \"38d10468-ff88-4cb3-a9b5-793f8df5acc5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.715239 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.715212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38d10468-ff88-4cb3-a9b5-793f8df5acc5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9g244\" (UID: \"38d10468-ff88-4cb3-a9b5-793f8df5acc5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.716474 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.716457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2v6\" (UniqueName: \"kubernetes.io/projected/38d10468-ff88-4cb3-a9b5-793f8df5acc5-kube-api-access-9h2v6\") pod \"cert-manager-webhook-6486ffd997-9g244\" (UID: \"38d10468-ff88-4cb3-a9b5-793f8df5acc5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.866804 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.866728    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:23.999587 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:23.999565    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9g244"]
Jul 23 03:59:24.002307 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:59:24.002272    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38d10468_ff88_4cb3_a9b5_793f8df5acc5.slice/crio-1d08232163ee5a6aec2d27d8105eee25ef588a821c7a33186c2f887dea05b86f WatchSource:0}: Error finding container 1d08232163ee5a6aec2d27d8105eee25ef588a821c7a33186c2f887dea05b86f: Status 404 returned error can't find the container with id 1d08232163ee5a6aec2d27d8105eee25ef588a821c7a33186c2f887dea05b86f
Jul 23 03:59:24.965746 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:24.965703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244" event={"ID":"38d10468-ff88-4cb3-a9b5-793f8df5acc5","Type":"ContainerStarted","Data":"1d08232163ee5a6aec2d27d8105eee25ef588a821c7a33186c2f887dea05b86f"}
Jul 23 03:59:27.979333 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:27.979296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244" event={"ID":"38d10468-ff88-4cb3-a9b5-793f8df5acc5","Type":"ContainerStarted","Data":"8ff73469a010a4f4901ca30d874e66871e52dc6b6754bf9ea4cc14eca75a7c6c"}
Jul 23 03:59:27.979784 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:27.979429    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:28.014077 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:28.014021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244" podStartSLOduration=2.084716306 podStartE2EDuration="5.014002065s" totalImagesPullingTime="2.929285759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:59:28.013996645 +0000 UTC m=+228.733528256" watchObservedRunningTime="2026-07-23 03:59:28.014002065 +0000 UTC m=+228.733533676"
Jul 23 03:59:33.986356 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:33.986317    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9g244"
Jul 23 03:59:35.438503 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.438466    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lwjpk"]
Jul 23 03:59:35.441731 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.441715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.445183 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.445159    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-658m8\""
Jul 23 03:59:35.451887 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.451866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lwjpk"]
Jul 23 03:59:35.604209 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.604177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7282aebc-0488-498a-bfce-b95a943b559a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lwjpk\" (UID: \"7282aebc-0488-498a-bfce-b95a943b559a\") " pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.604344 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.604227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sv88\" (UniqueName: \"kubernetes.io/projected/7282aebc-0488-498a-bfce-b95a943b559a-kube-api-access-8sv88\") pod \"cert-manager-55b4d954d5-lwjpk\" (UID: \"7282aebc-0488-498a-bfce-b95a943b559a\") " pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.704678 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.704655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7282aebc-0488-498a-bfce-b95a943b559a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lwjpk\" (UID: \"7282aebc-0488-498a-bfce-b95a943b559a\") " pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.704764 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.704701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sv88\" (UniqueName: \"kubernetes.io/projected/7282aebc-0488-498a-bfce-b95a943b559a-kube-api-access-8sv88\") pod \"cert-manager-55b4d954d5-lwjpk\" (UID: \"7282aebc-0488-498a-bfce-b95a943b559a\") " pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.713484 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.713456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7282aebc-0488-498a-bfce-b95a943b559a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lwjpk\" (UID: \"7282aebc-0488-498a-bfce-b95a943b559a\") " pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.713672 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.713654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sv88\" (UniqueName: \"kubernetes.io/projected/7282aebc-0488-498a-bfce-b95a943b559a-kube-api-access-8sv88\") pod \"cert-manager-55b4d954d5-lwjpk\" (UID: \"7282aebc-0488-498a-bfce-b95a943b559a\") " pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.750641 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.750616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lwjpk"
Jul 23 03:59:35.877181 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:35.877153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lwjpk"]
Jul 23 03:59:35.879688 ip-10-0-131-0 kubenswrapper[2563]: W0723 03:59:35.879660    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7282aebc_0488_498a_bfce_b95a943b559a.slice/crio-5f3d8347172c3e61aa137dcf6f32931588727cdc07d97656e0115598c0e26ce7 WatchSource:0}: Error finding container 5f3d8347172c3e61aa137dcf6f32931588727cdc07d97656e0115598c0e26ce7: Status 404 returned error can't find the container with id 5f3d8347172c3e61aa137dcf6f32931588727cdc07d97656e0115598c0e26ce7
Jul 23 03:59:36.014557 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:36.014474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lwjpk" event={"ID":"7282aebc-0488-498a-bfce-b95a943b559a","Type":"ContainerStarted","Data":"471e87a495d37d8e6ff4c6dd54f9946a81d62f16b12c32e584bbfce20817a51c"}
Jul 23 03:59:36.014557 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:36.014525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lwjpk" event={"ID":"7282aebc-0488-498a-bfce-b95a943b559a","Type":"ContainerStarted","Data":"5f3d8347172c3e61aa137dcf6f32931588727cdc07d97656e0115598c0e26ce7"}
Jul 23 03:59:36.032374 ip-10-0-131-0 kubenswrapper[2563]: I0723 03:59:36.032332    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lwjpk" podStartSLOduration=1.0323137039999999 podStartE2EDuration="1.032313704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:59:36.031245673 +0000 UTC m=+236.750777286" watchObservedRunningTime="2026-07-23 03:59:36.032313704 +0000 UTC m=+236.751845313"
Jul 23 04:00:01.697206 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.697131    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"]
Jul 23 04:00:01.704166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.704141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.708747 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.708730    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 04:00:01.709187 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.709173    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 04:00:01.710818 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.710797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 04:00:01.711054 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.711038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 04:00:01.713107 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.713091    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2clxt\""
Jul 23 04:00:01.716001 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.715984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttpwn\" (UniqueName: \"kubernetes.io/projected/841f5b8d-7249-4352-bffa-6c27f35e34d6-kube-api-access-ttpwn\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.716076 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.716013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/841f5b8d-7249-4352-bffa-6c27f35e34d6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.716076 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.716036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/841f5b8d-7249-4352-bffa-6c27f35e34d6-tmp\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.716149 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.716078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/841f5b8d-7249-4352-bffa-6c27f35e34d6-manifests\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.716149 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.716134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/841f5b8d-7249-4352-bffa-6c27f35e34d6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.774260 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.774228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"]
Jul 23 04:00:01.817166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.817133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/841f5b8d-7249-4352-bffa-6c27f35e34d6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.817349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.817177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttpwn\" (UniqueName: \"kubernetes.io/projected/841f5b8d-7249-4352-bffa-6c27f35e34d6-kube-api-access-ttpwn\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.817349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.817201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/841f5b8d-7249-4352-bffa-6c27f35e34d6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.817349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.817226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/841f5b8d-7249-4352-bffa-6c27f35e34d6-tmp\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.817349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.817251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/841f5b8d-7249-4352-bffa-6c27f35e34d6-manifests\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.817796 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.817770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/841f5b8d-7249-4352-bffa-6c27f35e34d6-manifests\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.819657 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.819630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/841f5b8d-7249-4352-bffa-6c27f35e34d6-tmp\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.819771 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.819753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/841f5b8d-7249-4352-bffa-6c27f35e34d6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.819912 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.819892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/841f5b8d-7249-4352-bffa-6c27f35e34d6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.835873 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.835851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttpwn\" (UniqueName: \"kubernetes.io/projected/841f5b8d-7249-4352-bffa-6c27f35e34d6-kube-api-access-ttpwn\") pod \"opendatahub-operator-controller-manager-5775946f86-4ktqs\" (UID: \"841f5b8d-7249-4352-bffa-6c27f35e34d6\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:01.882062 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.882032    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"]
Jul 23 04:00:01.885135 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.885120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:01.888269 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.888240    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 04:00:01.888434 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.888324    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r57g6\""
Jul 23 04:00:01.888609 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.888586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 04:00:01.888707 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.888679    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 04:00:01.888768 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.888706    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 04:00:01.889198 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.889181    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 04:00:01.902111 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.902086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"]
Jul 23 04:00:01.917739 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.917713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/596557b9-e5e1-47e8-b0e0-220b0b227715-manager-config\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:01.917935 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.917786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596557b9-e5e1-47e8-b0e0-220b0b227715-cert\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:01.917935 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.917809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnkdz\" (UniqueName: \"kubernetes.io/projected/596557b9-e5e1-47e8-b0e0-220b0b227715-kube-api-access-fnkdz\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:01.917935 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:01.917873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/596557b9-e5e1-47e8-b0e0-220b0b227715-metrics-cert\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.015676 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.015588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:02.018466 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.018228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/596557b9-e5e1-47e8-b0e0-220b0b227715-manager-config\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.018466 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.018285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596557b9-e5e1-47e8-b0e0-220b0b227715-cert\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.018466 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.018312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnkdz\" (UniqueName: \"kubernetes.io/projected/596557b9-e5e1-47e8-b0e0-220b0b227715-kube-api-access-fnkdz\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.018466 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.018355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/596557b9-e5e1-47e8-b0e0-220b0b227715-metrics-cert\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.018956 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.018928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/596557b9-e5e1-47e8-b0e0-220b0b227715-manager-config\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.020963 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.020933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/596557b9-e5e1-47e8-b0e0-220b0b227715-metrics-cert\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.021070 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.020997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596557b9-e5e1-47e8-b0e0-220b0b227715-cert\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.030048 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.030027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnkdz\" (UniqueName: \"kubernetes.io/projected/596557b9-e5e1-47e8-b0e0-220b0b227715-kube-api-access-fnkdz\") pod \"lws-controller-manager-6776597644-jcbwn\" (UID: \"596557b9-e5e1-47e8-b0e0-220b0b227715\") " pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.158267 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.158223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"]
Jul 23 04:00:02.161319 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:00:02.161291    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841f5b8d_7249_4352_bffa_6c27f35e34d6.slice/crio-23fe93f3a6821db0723bbb39e93b5c2fd12d73d258ab5aea36e3a47bb7a52963 WatchSource:0}: Error finding container 23fe93f3a6821db0723bbb39e93b5c2fd12d73d258ab5aea36e3a47bb7a52963: Status 404 returned error can't find the container with id 23fe93f3a6821db0723bbb39e93b5c2fd12d73d258ab5aea36e3a47bb7a52963
Jul 23 04:00:02.195349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.195324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:02.335060 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:02.335030    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"]
Jul 23 04:00:02.338538 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:00:02.338513    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596557b9_e5e1_47e8_b0e0_220b0b227715.slice/crio-e04b8bfcde0d6c17dd9d6101cc13ff91b2c7f9629fc0767366ac411af5a57119 WatchSource:0}: Error finding container e04b8bfcde0d6c17dd9d6101cc13ff91b2c7f9629fc0767366ac411af5a57119: Status 404 returned error can't find the container with id e04b8bfcde0d6c17dd9d6101cc13ff91b2c7f9629fc0767366ac411af5a57119
Jul 23 04:00:03.127738 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:03.127662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs" event={"ID":"841f5b8d-7249-4352-bffa-6c27f35e34d6","Type":"ContainerStarted","Data":"23fe93f3a6821db0723bbb39e93b5c2fd12d73d258ab5aea36e3a47bb7a52963"}
Jul 23 04:00:03.129658 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:03.129627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn" event={"ID":"596557b9-e5e1-47e8-b0e0-220b0b227715","Type":"ContainerStarted","Data":"e04b8bfcde0d6c17dd9d6101cc13ff91b2c7f9629fc0767366ac411af5a57119"}
Jul 23 04:00:06.148617 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:06.148578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs" event={"ID":"841f5b8d-7249-4352-bffa-6c27f35e34d6","Type":"ContainerStarted","Data":"afaf6c397a972f889441e918b11862dea5621d818b23d21203043c03c8e09347"}
Jul 23 04:00:07.154511 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:07.154479    2563 generic.go:356] "Generic (PLEG): container finished" podID="841f5b8d-7249-4352-bffa-6c27f35e34d6" containerID="afaf6c397a972f889441e918b11862dea5621d818b23d21203043c03c8e09347" exitCode=0
Jul 23 04:00:07.154511 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:07.154517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs" event={"ID":"841f5b8d-7249-4352-bffa-6c27f35e34d6","Type":"ContainerDied","Data":"afaf6c397a972f889441e918b11862dea5621d818b23d21203043c03c8e09347"}
Jul 23 04:00:08.160265 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:08.160230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs" event={"ID":"841f5b8d-7249-4352-bffa-6c27f35e34d6","Type":"ContainerStarted","Data":"6d0334a0d8415b1ea4b131d23ea51b5a7c34ca20a12e05927a0e3b0deceec377"}
Jul 23 04:00:08.160681 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:08.160471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:08.182646 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:08.182589    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs" podStartSLOduration=2.427186335 podStartE2EDuration="7.182566627s" totalImagesPullingTime="3.748676798s" totalInitContainerRuntime="1.006703494s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:08.181108822 +0000 UTC m=+268.900640431" watchObservedRunningTime="2026-07-23 04:00:08.182566627 +0000 UTC m=+268.902098238"
Jul 23 04:00:19.167438 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:19.167382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-4ktqs"
Jul 23 04:00:22.328553 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.328517    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"]
Jul 23 04:00:22.340270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.340234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.347031 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.346993    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 04:00:22.348331 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.348182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 04:00:22.348331 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.348215    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 04:00:22.348331 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.348224    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 04:00:22.348331 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.348241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fjd7q\""
Jul 23 04:00:22.362285 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.362255    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"]
Jul 23 04:00:22.407225 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.407183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh5jp\" (UniqueName: \"kubernetes.io/projected/6bb86030-2683-4b17-bd80-d07ed1845dfd-kube-api-access-sh5jp\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.407429 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.407254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb86030-2683-4b17-bd80-d07ed1845dfd-tls-certs\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.407429 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.407327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bb86030-2683-4b17-bd80-d07ed1845dfd-tmp\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.508362 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.508321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb86030-2683-4b17-bd80-d07ed1845dfd-tls-certs\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.508572 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.508404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bb86030-2683-4b17-bd80-d07ed1845dfd-tmp\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.508572 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.508512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh5jp\" (UniqueName: \"kubernetes.io/projected/6bb86030-2683-4b17-bd80-d07ed1845dfd-kube-api-access-sh5jp\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.511640 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.511399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb86030-2683-4b17-bd80-d07ed1845dfd-tls-certs\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.511640 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.511593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bb86030-2683-4b17-bd80-d07ed1845dfd-tmp\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.526520 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.526469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh5jp\" (UniqueName: \"kubernetes.io/projected/6bb86030-2683-4b17-bd80-d07ed1845dfd-kube-api-access-sh5jp\") pod \"kube-auth-proxy-c76cc57cd-4zmtr\" (UID: \"6bb86030-2683-4b17-bd80-d07ed1845dfd\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.653929 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.653852    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"
Jul 23 04:00:22.909833 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:22.909812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr"]
Jul 23 04:00:22.912875 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:00:22.912845    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb86030_2683_4b17_bd80_d07ed1845dfd.slice/crio-2b2265c9572aaca45a5ad58a031297e6c816860e7f6e36f4de5b2c8bd685c5d4 WatchSource:0}: Error finding container 2b2265c9572aaca45a5ad58a031297e6c816860e7f6e36f4de5b2c8bd685c5d4: Status 404 returned error can't find the container with id 2b2265c9572aaca45a5ad58a031297e6c816860e7f6e36f4de5b2c8bd685c5d4
Jul 23 04:00:23.231928 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:23.231882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn" event={"ID":"596557b9-e5e1-47e8-b0e0-220b0b227715","Type":"ContainerStarted","Data":"464aad0fbe9d58b58d3d72efb222bd80946bb1e8930b4f1ce65ed3a35ba5f5d0"}
Jul 23 04:00:23.232125 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:23.231992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:23.233060 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:23.233036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr" event={"ID":"6bb86030-2683-4b17-bd80-d07ed1845dfd","Type":"ContainerStarted","Data":"2b2265c9572aaca45a5ad58a031297e6c816860e7f6e36f4de5b2c8bd685c5d4"}
Jul 23 04:00:23.271116 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:23.271069    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn" podStartSLOduration=1.771260338 podStartE2EDuration="22.271052653s" totalImagesPullingTime="20.499792315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:23.269719859 +0000 UTC m=+283.989251471" watchObservedRunningTime="2026-07-23 04:00:23.271052653 +0000 UTC m=+283.990584262"
Jul 23 04:00:26.249426 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:26.249378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr" event={"ID":"6bb86030-2683-4b17-bd80-d07ed1845dfd","Type":"ContainerStarted","Data":"df685b4c95ff46313911eb756478910087c8838a4bef237027b0bdff82a420a7"}
Jul 23 04:00:26.267174 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:26.267121    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-4zmtr" podStartSLOduration=1.6751143160000002 podStartE2EDuration="4.267100634s" totalImagesPullingTime="2.591986318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:26.265775204 +0000 UTC m=+286.985306814" watchObservedRunningTime="2026-07-23 04:00:26.267100634 +0000 UTC m=+286.986632245"
Jul 23 04:00:34.239819 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:34.239784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6776597644-jcbwn"
Jul 23 04:00:39.626973 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:39.626942    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:00:39.627498 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:39.626942    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:00:39.634340 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:00:39.634315    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 04:02:14.400798 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.400759    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"]
Jul 23 04:02:14.404188 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.404168    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.406904 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.406878    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 04:02:14.406904 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.406896    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 04:02:14.407044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.406898    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 04:02:14.407695 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.407673    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pbwj7\""
Jul 23 04:02:14.407695 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.407679    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 04:02:14.414193 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.414174    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"]
Jul 23 04:02:14.500254 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.500226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.500394 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.500269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5t2m\" (UniqueName: \"kubernetes.io/projected/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-kube-api-access-p5t2m\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.500394 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.500295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.601425 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.601383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.601550 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.601458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5t2m\" (UniqueName: \"kubernetes.io/projected/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-kube-api-access-p5t2m\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.601550 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.601505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.601661 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:14.601545    2563 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 23 04:02:14.601661 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:14.601637    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-plugin-serving-cert podName:5771e8e3-ebd4-4ef8-87a5-dd360afc50e8 nodeName:}" failed. No retries permitted until 2026-07-23 04:02:15.101615859 +0000 UTC m=+395.821147450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-mxm69" (UID: "5771e8e3-ebd4-4ef8-87a5-dd360afc50e8") : secret "plugin-serving-cert" not found
Jul 23 04:02:14.602072 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.602054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:14.620771 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:14.620750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5t2m\" (UniqueName: \"kubernetes.io/projected/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-kube-api-access-p5t2m\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:15.106612 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:15.106578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:15.109073 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:15.109053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5771e8e3-ebd4-4ef8-87a5-dd360afc50e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mxm69\" (UID: \"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:15.313801 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:15.313755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"
Jul 23 04:02:15.444764 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:15.444732    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69"]
Jul 23 04:02:15.447964 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:02:15.447935    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5771e8e3_ebd4_4ef8_87a5_dd360afc50e8.slice/crio-b2a65a717a252429a19566480db508b9122eb380bd2678517ecd40d76b7c7b13 WatchSource:0}: Error finding container b2a65a717a252429a19566480db508b9122eb380bd2678517ecd40d76b7c7b13: Status 404 returned error can't find the container with id b2a65a717a252429a19566480db508b9122eb380bd2678517ecd40d76b7c7b13
Jul 23 04:02:15.449509 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:15.449490    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 04:02:15.753961 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:15.753931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69" event={"ID":"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8","Type":"ContainerStarted","Data":"b2a65a717a252429a19566480db508b9122eb380bd2678517ecd40d76b7c7b13"}
Jul 23 04:02:25.348471 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.348436    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"]
Jul 23 04:02:25.374342 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.374310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"]
Jul 23 04:02:25.374529 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.374468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.377150 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.377124    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qrd4w\""
Jul 23 04:02:25.413155 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.413124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9450065f-9ce5-4d1f-be3c-ef671291cce3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.413309 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.413182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t5b7\" (UniqueName: \"kubernetes.io/projected/9450065f-9ce5-4d1f-be3c-ef671291cce3-kube-api-access-9t5b7\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.514574 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.514521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9450065f-9ce5-4d1f-be3c-ef671291cce3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.514749 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.514582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t5b7\" (UniqueName: \"kubernetes.io/projected/9450065f-9ce5-4d1f-be3c-ef671291cce3-kube-api-access-9t5b7\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.514950 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.514933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9450065f-9ce5-4d1f-be3c-ef671291cce3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.529152 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.529123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t5b7\" (UniqueName: \"kubernetes.io/projected/9450065f-9ce5-4d1f-be3c-ef671291cce3-kube-api-access-9t5b7\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.614720 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.614635    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"]
Jul 23 04:02:25.614990 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.614974    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:25.621531 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.621504    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"]
Jul 23 04:02:25.639144 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.639114    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"]
Jul 23 04:02:25.644171 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.644149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.663974 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.663905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"]
Jul 23 04:02:25.716615 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.716364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d763f16-19b6-4461-baba-c0260daa8b36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9f4pc\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.716615 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.716478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfsfj\" (UniqueName: \"kubernetes.io/projected/7d763f16-19b6-4461-baba-c0260daa8b36-kube-api-access-gfsfj\") pod \"kuadrant-operator-controller-manager-55467d9f7-9f4pc\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.817637 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.817595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d763f16-19b6-4461-baba-c0260daa8b36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9f4pc\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.817824 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.817653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfsfj\" (UniqueName: \"kubernetes.io/projected/7d763f16-19b6-4461-baba-c0260daa8b36-kube-api-access-gfsfj\") pod \"kuadrant-operator-controller-manager-55467d9f7-9f4pc\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.817974 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.817955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d763f16-19b6-4461-baba-c0260daa8b36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9f4pc\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.827046 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.827021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfsfj\" (UniqueName: \"kubernetes.io/projected/7d763f16-19b6-4461-baba-c0260daa8b36-kube-api-access-gfsfj\") pod \"kuadrant-operator-controller-manager-55467d9f7-9f4pc\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:25.956958 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:25.956919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:37.016142 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:37.016119    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"]
Jul 23 04:02:37.019486 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:02:37.019462    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d763f16_19b6_4461_baba_c0260daa8b36.slice/crio-010d4b9d505e1823ad64de1253fafd316c584bb99b8edb8f990b980e72a2f641 WatchSource:0}: Error finding container 010d4b9d505e1823ad64de1253fafd316c584bb99b8edb8f990b980e72a2f641: Status 404 returned error can't find the container with id 010d4b9d505e1823ad64de1253fafd316c584bb99b8edb8f990b980e72a2f641
Jul 23 04:02:37.887999 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:37.887936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69" event={"ID":"5771e8e3-ebd4-4ef8-87a5-dd360afc50e8","Type":"ContainerStarted","Data":"424c3c8eb77e5ef659002e44cc81680872762d794a02cc184f2e5cf9e868ae18"}
Jul 23 04:02:37.890024 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:37.889995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc" event={"ID":"7d763f16-19b6-4461-baba-c0260daa8b36","Type":"ContainerStarted","Data":"010d4b9d505e1823ad64de1253fafd316c584bb99b8edb8f990b980e72a2f641"}
Jul 23 04:02:37.919163 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:37.919104    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mxm69" podStartSLOduration=2.416552759 podStartE2EDuration="23.919083773s" totalImagesPullingTime="21.502531014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:37.916116622 +0000 UTC m=+418.635648232" watchObservedRunningTime="2026-07-23 04:02:37.919083773 +0000 UTC m=+418.638615384"
Jul 23 04:02:39.414888 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:39.414839    2563 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-n8bkd_kuadrant-system_9450065f-9ce5-4d1f-be3c-ef671291cce3_0(b0b3503e155f8f1cdaf82ab13eb1ba8cceb8fecead3e5dbda19e18607191e678): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-n8bkd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 23 04:02:39.415236 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:39.414920    2563 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-n8bkd_kuadrant-system_9450065f-9ce5-4d1f-be3c-ef671291cce3_0(b0b3503e155f8f1cdaf82ab13eb1ba8cceb8fecead3e5dbda19e18607191e678): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-n8bkd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:39.899390 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:39.899356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:39.901757 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:39.901722    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" is forbidden: User \"system:node:ip-10-0-131-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-0.ec2.internal' and this object" podUID="9450065f-9ce5-4d1f-be3c-ef671291cce3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:39.908044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:39.908024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:39.912357 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:39.912327    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" is forbidden: User \"system:node:ip-10-0-131-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-0.ec2.internal' and this object" podUID="9450065f-9ce5-4d1f-be3c-ef671291cce3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:40.056837 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.056806    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9450065f-9ce5-4d1f-be3c-ef671291cce3-extensions-socket-volume\") pod \"9450065f-9ce5-4d1f-be3c-ef671291cce3\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") "
Jul 23 04:02:40.057046 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.056894    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t5b7\" (UniqueName: \"kubernetes.io/projected/9450065f-9ce5-4d1f-be3c-ef671291cce3-kube-api-access-9t5b7\") pod \"9450065f-9ce5-4d1f-be3c-ef671291cce3\" (UID: \"9450065f-9ce5-4d1f-be3c-ef671291cce3\") "
Jul 23 04:02:40.057158 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.057135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9450065f-9ce5-4d1f-be3c-ef671291cce3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9450065f-9ce5-4d1f-be3c-ef671291cce3" (UID: "9450065f-9ce5-4d1f-be3c-ef671291cce3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:02:40.059276 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.059247    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9450065f-9ce5-4d1f-be3c-ef671291cce3-kube-api-access-9t5b7" (OuterVolumeSpecName: "kube-api-access-9t5b7") pod "9450065f-9ce5-4d1f-be3c-ef671291cce3" (UID: "9450065f-9ce5-4d1f-be3c-ef671291cce3"). InnerVolumeSpecName "kube-api-access-9t5b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:02:40.158231 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.158159    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9450065f-9ce5-4d1f-be3c-ef671291cce3-extensions-socket-volume\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:02:40.158231 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.158192    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t5b7\" (UniqueName: \"kubernetes.io/projected/9450065f-9ce5-4d1f-be3c-ef671291cce3-kube-api-access-9t5b7\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:02:40.904663 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:40.904629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:40.907611 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:40.907577    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" is forbidden: User \"system:node:ip-10-0-131-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-0.ec2.internal' and this object" podUID="9450065f-9ce5-4d1f-be3c-ef671291cce3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:40.915120 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:02:40.915087    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8bkd\" is forbidden: User \"system:node:ip-10-0-131-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-0.ec2.internal' and this object" podUID="9450065f-9ce5-4d1f-be3c-ef671291cce3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8bkd"
Jul 23 04:02:41.749911 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:41.749880    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9450065f-9ce5-4d1f-be3c-ef671291cce3" path="/var/lib/kubelet/pods/9450065f-9ce5-4d1f-be3c-ef671291cce3/volumes"
Jul 23 04:02:41.911140 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:41.911057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc" event={"ID":"7d763f16-19b6-4461-baba-c0260daa8b36","Type":"ContainerStarted","Data":"1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792"}
Jul 23 04:02:41.911521 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:41.911276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:02:41.932166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:41.932121    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc" podStartSLOduration=12.377465411 podStartE2EDuration="16.932107052s" totalImagesPullingTime="4.554641641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:41.930323729 +0000 UTC m=+422.649855343" watchObservedRunningTime="2026-07-23 04:02:41.932107052 +0000 UTC m=+422.651638662"
Jul 23 04:02:52.918982 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:02:52.918950    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:03:05.168560 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.168462    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"]
Jul 23 04:03:05.168975 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.168707    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc" podUID="7d763f16-19b6-4461-baba-c0260daa8b36" containerName="manager" containerID="cri-o://1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792" gracePeriod=10
Jul 23 04:03:05.948927 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.948901    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:03:05.984534 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.984511    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d763f16-19b6-4461-baba-c0260daa8b36-extensions-socket-volume\") pod \"7d763f16-19b6-4461-baba-c0260daa8b36\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") "
Jul 23 04:03:05.984655 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.984564    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfsfj\" (UniqueName: \"kubernetes.io/projected/7d763f16-19b6-4461-baba-c0260daa8b36-kube-api-access-gfsfj\") pod \"7d763f16-19b6-4461-baba-c0260daa8b36\" (UID: \"7d763f16-19b6-4461-baba-c0260daa8b36\") "
Jul 23 04:03:05.984976 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.984949    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d763f16-19b6-4461-baba-c0260daa8b36-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7d763f16-19b6-4461-baba-c0260daa8b36" (UID: "7d763f16-19b6-4461-baba-c0260daa8b36"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:03:05.987004 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:05.986980    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d763f16-19b6-4461-baba-c0260daa8b36-kube-api-access-gfsfj" (OuterVolumeSpecName: "kube-api-access-gfsfj") pod "7d763f16-19b6-4461-baba-c0260daa8b36" (UID: "7d763f16-19b6-4461-baba-c0260daa8b36"). InnerVolumeSpecName "kube-api-access-gfsfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:03:06.021881 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.021854    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d763f16-19b6-4461-baba-c0260daa8b36" containerID="1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792" exitCode=0
Jul 23 04:03:06.021973 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.021895    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"
Jul 23 04:03:06.021973 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.021941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc" event={"ID":"7d763f16-19b6-4461-baba-c0260daa8b36","Type":"ContainerDied","Data":"1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792"}
Jul 23 04:03:06.022048 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.021994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc" event={"ID":"7d763f16-19b6-4461-baba-c0260daa8b36","Type":"ContainerDied","Data":"010d4b9d505e1823ad64de1253fafd316c584bb99b8edb8f990b980e72a2f641"}
Jul 23 04:03:06.022048 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.022016    2563 scope.go:117] "RemoveContainer" containerID="1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792"
Jul 23 04:03:06.031391 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.031377    2563 scope.go:117] "RemoveContainer" containerID="1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792"
Jul 23 04:03:06.031699 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:03:06.031682    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792\": container with ID starting with 1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792 not found: ID does not exist" containerID="1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792"
Jul 23 04:03:06.031745 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.031708    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792"} err="failed to get container status \"1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792\": rpc error: code = NotFound desc = could not find container \"1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792\": container with ID starting with 1eae14b3251a3f2832dc956400e3cdf08a047993be06c6c7af8bae01f1a21792 not found: ID does not exist"
Jul 23 04:03:06.046494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.046445    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"]
Jul 23 04:03:06.059916 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.059895    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9f4pc"]
Jul 23 04:03:06.086035 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.086013    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d763f16-19b6-4461-baba-c0260daa8b36-extensions-socket-volume\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:03:06.086035 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:06.086032    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfsfj\" (UniqueName: \"kubernetes.io/projected/7d763f16-19b6-4461-baba-c0260daa8b36-kube-api-access-gfsfj\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:03:07.750275 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:07.750242    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d763f16-19b6-4461-baba-c0260daa8b36" path="/var/lib/kubelet/pods/7d763f16-19b6-4461-baba-c0260daa8b36/volumes"
Jul 23 04:03:14.732498 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.732460    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:03:14.733055 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.733032    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d763f16-19b6-4461-baba-c0260daa8b36" containerName="manager"
Jul 23 04:03:14.733055 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.733051    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d763f16-19b6-4461-baba-c0260daa8b36" containerName="manager"
Jul 23 04:03:14.733179 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.733166    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d763f16-19b6-4461-baba-c0260daa8b36" containerName="manager"
Jul 23 04:03:14.737551 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.737531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:14.741213 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.741194    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 04:03:14.757393 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.757374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:03:14.812882 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.812840    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:03:14.866574 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.866547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b633855a-5919-405e-9866-16a6d49e5c4d-config-file\") pod \"limitador-limitador-54d6d988d5-l884s\" (UID: \"b633855a-5919-405e-9866-16a6d49e5c4d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:14.866713 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.866598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxcgl\" (UniqueName: \"kubernetes.io/projected/b633855a-5919-405e-9866-16a6d49e5c4d-kube-api-access-gxcgl\") pod \"limitador-limitador-54d6d988d5-l884s\" (UID: \"b633855a-5919-405e-9866-16a6d49e5c4d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:14.967817 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.967784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b633855a-5919-405e-9866-16a6d49e5c4d-config-file\") pod \"limitador-limitador-54d6d988d5-l884s\" (UID: \"b633855a-5919-405e-9866-16a6d49e5c4d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:14.967952 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.967835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxcgl\" (UniqueName: \"kubernetes.io/projected/b633855a-5919-405e-9866-16a6d49e5c4d-kube-api-access-gxcgl\") pod \"limitador-limitador-54d6d988d5-l884s\" (UID: \"b633855a-5919-405e-9866-16a6d49e5c4d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:14.968491 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.968471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b633855a-5919-405e-9866-16a6d49e5c4d-config-file\") pod \"limitador-limitador-54d6d988d5-l884s\" (UID: \"b633855a-5919-405e-9866-16a6d49e5c4d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:14.978061 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:14.978030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxcgl\" (UniqueName: \"kubernetes.io/projected/b633855a-5919-405e-9866-16a6d49e5c4d-kube-api-access-gxcgl\") pod \"limitador-limitador-54d6d988d5-l884s\" (UID: \"b633855a-5919-405e-9866-16a6d49e5c4d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:15.047578 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.047511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:15.182694 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.182672    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:03:15.184794 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:03:15.184768    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb633855a_5919_405e_9866_16a6d49e5c4d.slice/crio-6243425f2bee82427ca1643230e1b9de2fbf946ba8ad6183b7a0403a1532d498 WatchSource:0}: Error finding container 6243425f2bee82427ca1643230e1b9de2fbf946ba8ad6183b7a0403a1532d498: Status 404 returned error can't find the container with id 6243425f2bee82427ca1643230e1b9de2fbf946ba8ad6183b7a0403a1532d498
Jul 23 04:03:15.503797 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.503763    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-vz5jk"]
Jul 23 04:03:15.508146 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.508126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:15.510519 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.510491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mrzjq\""
Jul 23 04:03:15.521699 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.521679    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vz5jk"]
Jul 23 04:03:15.573082 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.573052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztbtz\" (UniqueName: \"kubernetes.io/projected/728a3a0d-fda6-46dc-b7a5-6247063c2527-kube-api-access-ztbtz\") pod \"authorino-75d556bd77-vz5jk\" (UID: \"728a3a0d-fda6-46dc-b7a5-6247063c2527\") " pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:15.674465 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.674400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztbtz\" (UniqueName: \"kubernetes.io/projected/728a3a0d-fda6-46dc-b7a5-6247063c2527-kube-api-access-ztbtz\") pod \"authorino-75d556bd77-vz5jk\" (UID: \"728a3a0d-fda6-46dc-b7a5-6247063c2527\") " pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:15.689267 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.689215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztbtz\" (UniqueName: \"kubernetes.io/projected/728a3a0d-fda6-46dc-b7a5-6247063c2527-kube-api-access-ztbtz\") pod \"authorino-75d556bd77-vz5jk\" (UID: \"728a3a0d-fda6-46dc-b7a5-6247063c2527\") " pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:15.818284 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.818199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:15.989505 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:15.989479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vz5jk"]
Jul 23 04:03:15.992477 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:03:15.992448    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod728a3a0d_fda6_46dc_b7a5_6247063c2527.slice/crio-1aea7472fd0afab1e0dfe2ace4cf456fecaa8c64f6af0ec111cc2cbe5c005b8c WatchSource:0}: Error finding container 1aea7472fd0afab1e0dfe2ace4cf456fecaa8c64f6af0ec111cc2cbe5c005b8c: Status 404 returned error can't find the container with id 1aea7472fd0afab1e0dfe2ace4cf456fecaa8c64f6af0ec111cc2cbe5c005b8c
Jul 23 04:03:16.070077 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:16.069975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vz5jk" event={"ID":"728a3a0d-fda6-46dc-b7a5-6247063c2527","Type":"ContainerStarted","Data":"1aea7472fd0afab1e0dfe2ace4cf456fecaa8c64f6af0ec111cc2cbe5c005b8c"}
Jul 23 04:03:16.071451 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:16.071424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s" event={"ID":"b633855a-5919-405e-9866-16a6d49e5c4d","Type":"ContainerStarted","Data":"6243425f2bee82427ca1643230e1b9de2fbf946ba8ad6183b7a0403a1532d498"}
Jul 23 04:03:18.085556 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:18.085465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s" event={"ID":"b633855a-5919-405e-9866-16a6d49e5c4d","Type":"ContainerStarted","Data":"4bdb739c28f850f21013aab42d4e8d60fed3db77a16fc5d382310f7946eff879"}
Jul 23 04:03:18.085955 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:18.085561    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:18.104913 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:18.104860    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s" podStartSLOduration=1.500764046 podStartE2EDuration="4.104829556s" totalImagesPullingTime="2.60406551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:18.101744867 +0000 UTC m=+458.821276477" watchObservedRunningTime="2026-07-23 04:03:18.104829556 +0000 UTC m=+458.824361168"
Jul 23 04:03:20.096210 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:20.096171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vz5jk" event={"ID":"728a3a0d-fda6-46dc-b7a5-6247063c2527","Type":"ContainerStarted","Data":"a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8"}
Jul 23 04:03:20.113191 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:20.113144    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-vz5jk" podStartSLOduration=1.208713296 podStartE2EDuration="5.113130701s" totalImagesPullingTime="3.904417405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:20.110850452 +0000 UTC m=+460.830382059" watchObservedRunningTime="2026-07-23 04:03:20.113130701 +0000 UTC m=+460.832662310"
Jul 23 04:03:29.091401 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:29.091371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-l884s"
Jul 23 04:03:44.864438 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:44.864393    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vz5jk"]
Jul 23 04:03:44.864890 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:44.864560    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-vz5jk" podUID="728a3a0d-fda6-46dc-b7a5-6247063c2527" containerName="authorino" containerID="cri-o://a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8" gracePeriod=30
Jul 23 04:03:45.112251 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.112229    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:45.212372 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.212345    2563 generic.go:356] "Generic (PLEG): container finished" podID="728a3a0d-fda6-46dc-b7a5-6247063c2527" containerID="a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8" exitCode=0
Jul 23 04:03:45.212558 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.212385    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vz5jk"
Jul 23 04:03:45.212558 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.212437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vz5jk" event={"ID":"728a3a0d-fda6-46dc-b7a5-6247063c2527","Type":"ContainerDied","Data":"a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8"}
Jul 23 04:03:45.212558 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.212475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vz5jk" event={"ID":"728a3a0d-fda6-46dc-b7a5-6247063c2527","Type":"ContainerDied","Data":"1aea7472fd0afab1e0dfe2ace4cf456fecaa8c64f6af0ec111cc2cbe5c005b8c"}
Jul 23 04:03:45.212558 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.212492    2563 scope.go:117] "RemoveContainer" containerID="a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8"
Jul 23 04:03:45.220919 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.220902    2563 scope.go:117] "RemoveContainer" containerID="a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8"
Jul 23 04:03:45.221177 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:03:45.221160    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8\": container with ID starting with a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8 not found: ID does not exist" containerID="a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8"
Jul 23 04:03:45.221220 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.221187    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8"} err="failed to get container status \"a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8\": rpc error: code = NotFound desc = could not find container \"a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8\": container with ID starting with a56d9f6cb7279af1a62d7c9e03b3c25f2a59ecc63a20227a76fa4805670a65e8 not found: ID does not exist"
Jul 23 04:03:45.245375 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.245352    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztbtz\" (UniqueName: \"kubernetes.io/projected/728a3a0d-fda6-46dc-b7a5-6247063c2527-kube-api-access-ztbtz\") pod \"728a3a0d-fda6-46dc-b7a5-6247063c2527\" (UID: \"728a3a0d-fda6-46dc-b7a5-6247063c2527\") "
Jul 23 04:03:45.247588 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.247569    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/728a3a0d-fda6-46dc-b7a5-6247063c2527-kube-api-access-ztbtz" (OuterVolumeSpecName: "kube-api-access-ztbtz") pod "728a3a0d-fda6-46dc-b7a5-6247063c2527" (UID: "728a3a0d-fda6-46dc-b7a5-6247063c2527"). InnerVolumeSpecName "kube-api-access-ztbtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:03:45.346144 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.346113    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztbtz\" (UniqueName: \"kubernetes.io/projected/728a3a0d-fda6-46dc-b7a5-6247063c2527-kube-api-access-ztbtz\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:03:45.534922 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.534895    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vz5jk"]
Jul 23 04:03:45.539000 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.538978    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vz5jk"]
Jul 23 04:03:45.751277 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:03:45.751238    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="728a3a0d-fda6-46dc-b7a5-6247063c2527" path="/var/lib/kubelet/pods/728a3a0d-fda6-46dc-b7a5-6247063c2527/volumes"
Jul 23 04:04:25.815260 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:04:25.815229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:04:47.902729 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:04:47.902681    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:05:01.388024 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.387990    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"]
Jul 23 04:05:01.388577 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.388561    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="728a3a0d-fda6-46dc-b7a5-6247063c2527" containerName="authorino"
Jul 23 04:05:01.388623 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.388581    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="728a3a0d-fda6-46dc-b7a5-6247063c2527" containerName="authorino"
Jul 23 04:05:01.388697 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.388685    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="728a3a0d-fda6-46dc-b7a5-6247063c2527" containerName="authorino"
Jul 23 04:05:01.400974 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.400952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.401360 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.401325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"]
Jul 23 04:05:01.404946 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.404920    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 04:05:01.405066 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.404992    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 04:05:01.405204 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.405190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tznxk\""
Jul 23 04:05:01.406170 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.406144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 23 04:05:01.497728 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.497693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqh6j\" (UniqueName: \"kubernetes.io/projected/8cf01197-ef21-405c-8870-75057f3ecaef-kube-api-access-fqh6j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.497889 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.497735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.497889 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.497760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.497889 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.497829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.497889 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.497858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.498024 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.497922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf01197-ef21-405c-8870-75057f3ecaef-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.599389 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf01197-ef21-405c-8870-75057f3ecaef-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.599600 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqh6j\" (UniqueName: \"kubernetes.io/projected/8cf01197-ef21-405c-8870-75057f3ecaef-kube-api-access-fqh6j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.599600 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.599600 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.599600 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.599816 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.600021 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.599995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.600163 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.600025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.600163 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.600124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.601929 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.601908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf01197-ef21-405c-8870-75057f3ecaef-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.602056 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.602038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf01197-ef21-405c-8870-75057f3ecaef-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.608134 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.608114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqh6j\" (UniqueName: \"kubernetes.io/projected/8cf01197-ef21-405c-8870-75057f3ecaef-kube-api-access-fqh6j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q5b6w\" (UID: \"8cf01197-ef21-405c-8870-75057f3ecaef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.712461 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.712437    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:01.848555 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:01.848532    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"]
Jul 23 04:05:01.850799 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:05:01.850769    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf01197_ef21_405c_8870_75057f3ecaef.slice/crio-4c45b66b464a221a505ad2bc72a916420d37507ce4b9bfb2fc74edf5a75c0c45 WatchSource:0}: Error finding container 4c45b66b464a221a505ad2bc72a916420d37507ce4b9bfb2fc74edf5a75c0c45: Status 404 returned error can't find the container with id 4c45b66b464a221a505ad2bc72a916420d37507ce4b9bfb2fc74edf5a75c0c45
Jul 23 04:05:02.200498 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:02.200470    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:05:02.565516 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:02.565402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w" event={"ID":"8cf01197-ef21-405c-8870-75057f3ecaef","Type":"ContainerStarted","Data":"4c45b66b464a221a505ad2bc72a916420d37507ce4b9bfb2fc74edf5a75c0c45"}
Jul 23 04:05:04.577079 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:04.577042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w" event={"ID":"8cf01197-ef21-405c-8870-75057f3ecaef","Type":"ContainerStarted","Data":"93ef5ed69163f3f4f464665cfce0dd74d5ee2973fa2671d23f42e0f82fff2014"}
Jul 23 04:05:04.595447 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:04.595382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w" podStartSLOduration=1.9042279789999998 podStartE2EDuration="3.595362092s" totalImagesPullingTime="1.691134113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:05:04.594803954 +0000 UTC m=+565.314335567" watchObservedRunningTime="2026-07-23 04:05:04.595362092 +0000 UTC m=+565.314893701"
Jul 23 04:05:11.299727 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.299685    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"]
Jul 23 04:05:11.303836 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.303812    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.306502 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.306468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 23 04:05:11.316652 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.316628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"]
Jul 23 04:05:11.378907 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.378871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.379080 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.378938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.379080 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.378976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.379080 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.378994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.379080 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.379009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6242d091-0c2f-4de6-9dc3-c1148e2d236c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.379080 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.379067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfmbn\" (UniqueName: \"kubernetes.io/projected/6242d091-0c2f-4de6-9dc3-c1148e2d236c-kube-api-access-nfmbn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.479812 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.479762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.479812 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.479815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.479985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.479839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.479985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.479853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.479985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.479868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6242d091-0c2f-4de6-9dc3-c1148e2d236c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.479985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.479887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfmbn\" (UniqueName: \"kubernetes.io/projected/6242d091-0c2f-4de6-9dc3-c1148e2d236c-kube-api-access-nfmbn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.480269 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.480200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.480320 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.480264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.480356 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.480341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.482160 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.482139    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6242d091-0c2f-4de6-9dc3-c1148e2d236c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.482402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.482384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6242d091-0c2f-4de6-9dc3-c1148e2d236c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.489209 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.489184    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfmbn\" (UniqueName: \"kubernetes.io/projected/6242d091-0c2f-4de6-9dc3-c1148e2d236c-kube-api-access-nfmbn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5cw2s\" (UID: \"6242d091-0c2f-4de6-9dc3-c1148e2d236c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.614303 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.614224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:11.713454 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.712724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:11.713454 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.713196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:11.725909 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.725887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:11.742791 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:11.742763    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"]
Jul 23 04:05:11.746652 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:05:11.746630    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6242d091_0c2f_4de6_9dc3_c1148e2d236c.slice/crio-92584b61bba5c0bdb20f4b37532949975d16d9722be7533698cc82b429db2a75 WatchSource:0}: Error finding container 92584b61bba5c0bdb20f4b37532949975d16d9722be7533698cc82b429db2a75: Status 404 returned error can't find the container with id 92584b61bba5c0bdb20f4b37532949975d16d9722be7533698cc82b429db2a75
Jul 23 04:05:12.389402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:12.389328    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:05:12.614726 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:12.614695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s" event={"ID":"6242d091-0c2f-4de6-9dc3-c1148e2d236c","Type":"ContainerStarted","Data":"4c69f5d21421dc062582f59c3254b9cbfffbd48df3f3ac799c7154ed3035c2b3"}
Jul 23 04:05:12.614887 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:12.614745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s" event={"ID":"6242d091-0c2f-4de6-9dc3-c1148e2d236c","Type":"ContainerStarted","Data":"92584b61bba5c0bdb20f4b37532949975d16d9722be7533698cc82b429db2a75"}
Jul 23 04:05:12.626258 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:12.626239    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q5b6w"
Jul 23 04:05:12.634127 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:12.634082    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s" podStartSLOduration=1.44054648 podStartE2EDuration="1.634067757s" totalImagesPullingTime="193.521277ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:05:12.633818225 +0000 UTC m=+573.353349837" watchObservedRunningTime="2026-07-23 04:05:12.634067757 +0000 UTC m=+573.353599369"
Jul 23 04:05:21.615327 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:21.615287    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:21.615679 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:21.615358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:21.627703 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:21.627681    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:21.668519 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:21.668489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5cw2s"
Jul 23 04:05:24.993042 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:24.993007    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:05:28.199591 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.199551    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"]
Jul 23 04:05:28.203841 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.203817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.206162 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.206143    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 23 04:05:28.216882 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.216860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"]
Jul 23 04:05:28.335403 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.335373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.335587 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.335426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.335587 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.335497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.335587 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.335533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzqvj\" (UniqueName: \"kubernetes.io/projected/adb0b845-04d6-445e-ac9c-262bed6ca08c-kube-api-access-wzqvj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.335693 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.335637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.335693 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.335665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb0b845-04d6-445e-ac9c-262bed6ca08c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.436785 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.436750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.436785 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.436784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb0b845-04d6-445e-ac9c-262bed6ca08c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.436972 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.436836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.436972 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.436863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.436972 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.436891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.436972 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.436921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzqvj\" (UniqueName: \"kubernetes.io/projected/adb0b845-04d6-445e-ac9c-262bed6ca08c-kube-api-access-wzqvj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.437211 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.437181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.437339 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.437192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.437339 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.437278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.439222 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.439196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb0b845-04d6-445e-ac9c-262bed6ca08c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.439482 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.439466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb0b845-04d6-445e-ac9c-262bed6ca08c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.445512 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.445488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzqvj\" (UniqueName: \"kubernetes.io/projected/adb0b845-04d6-445e-ac9c-262bed6ca08c-kube-api-access-wzqvj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d8v84\" (UID: \"adb0b845-04d6-445e-ac9c-262bed6ca08c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.514448 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.514361    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:28.646402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.646380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"]
Jul 23 04:05:28.648456 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:05:28.648426    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadb0b845_04d6_445e_ac9c_262bed6ca08c.slice/crio-3edc3c08b38cd9659710ef53d6f4fa2a1556c894aa109f5bff0270b0870c4e4c WatchSource:0}: Error finding container 3edc3c08b38cd9659710ef53d6f4fa2a1556c894aa109f5bff0270b0870c4e4c: Status 404 returned error can't find the container with id 3edc3c08b38cd9659710ef53d6f4fa2a1556c894aa109f5bff0270b0870c4e4c
Jul 23 04:05:28.694231 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:28.694202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84" event={"ID":"adb0b845-04d6-445e-ac9c-262bed6ca08c","Type":"ContainerStarted","Data":"3edc3c08b38cd9659710ef53d6f4fa2a1556c894aa109f5bff0270b0870c4e4c"}
Jul 23 04:05:29.700499 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:29.700457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84" event={"ID":"adb0b845-04d6-445e-ac9c-262bed6ca08c","Type":"ContainerStarted","Data":"d88f15c877cf57f43fb67dd64e713ac0eef0dcdec52ce5b6d955a1695e39f5f7"}
Jul 23 04:05:29.720925 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:29.720874    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84" podStartSLOduration=1.513190513 podStartE2EDuration="1.720848064s" totalImagesPullingTime="207.657551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:05:29.718968643 +0000 UTC m=+590.438500252" watchObservedRunningTime="2026-07-23 04:05:29.720848064 +0000 UTC m=+590.440379673"
Jul 23 04:05:30.296006 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:30.295972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:05:38.514570 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:38.514527    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:38.515031 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:38.514604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:38.526719 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:38.526695    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:38.756578 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:38.756549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d8v84"
Jul 23 04:05:39.656244 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:39.656215    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:05:39.658366 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:39.658341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:05:40.100001 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:05:40.099967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:07:00.100373 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:07:00.100335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:07:09.092381 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:07:09.092341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:07:19.394840 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:07:19.394805    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:08:59.895285 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:08:59.895246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:09:10.596111 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:09:10.596029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:09:19.595463 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:09:19.595364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:09:29.101531 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:09:29.101498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:09:49.700288 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:09:49.700253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:09:59.899081 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:09:59.899048    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:10:08.793504 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:08.793467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:10:17.989883 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:17.989853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:10:27.997977 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:27.997946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:10:37.888279 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:37.888198    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:10:39.685732 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:39.685704    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:10:39.693053 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:39.693030    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:10:48.930032 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:48.929997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:10:58.595093 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:10:58.595061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:11:21.104495 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:11:21.104459    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:11:31.202627 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:11:31.202590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:11:55.215230 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:11:55.215197    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:12:05.794598 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:12:05.794519    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:12:21.017690 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:12:21.017654    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:12:31.513435 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:12:31.513389    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:12:40.647498 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:12:40.647462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:12:51.793659 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:12:51.793622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:13:00.691875 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:13:00.691844    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:14:47.294216 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:14:47.294179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:14:59.195143 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:14:59.195107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:15:07.794967 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:07.794884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:15:19.888515 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:19.888479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:15:32.306876 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:32.306838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:15:39.722382 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:39.722356    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:15:39.724994 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:39.724975    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:15:41.795655 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:41.795624    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:15:50.998944 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:15:50.998908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:16:02.994842 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:16:02.994807    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:16:12.098776 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:16:12.098742    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:16:21.297305 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:16:21.297267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:16:31.594770 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:16:31.594690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:16:56.088474 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:16:56.088438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:17:03.501443 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:03.501390    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:17:16.704876 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.704834    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"]
Jul 23 04:17:16.708378 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.708361    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.710873 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.710851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-819be5df-kserve-self-signed-certs\""
Jul 23 04:17:16.727044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.727018    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"]
Jul 23 04:17:16.770622 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.770592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr5pr\" (UniqueName: \"kubernetes.io/projected/b9cc6b3b-7847-436f-98e0-83190948a8c5-kube-api-access-qr5pr\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.770796 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.770637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cc6b3b-7847-436f-98e0-83190948a8c5-tls-certs\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.770796 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.770656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-tmp-dir\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.770796 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.770767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-home\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.770796 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.770791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-dshm\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.770960 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.770811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-model-cache\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.871490 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.871454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cc6b3b-7847-436f-98e0-83190948a8c5-tls-certs\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.871671 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.871498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-tmp-dir\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.871671 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.871555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-home\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.871671 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.871569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-dshm\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.871671 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.871583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-model-cache\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.871671 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.871653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr5pr\" (UniqueName: \"kubernetes.io/projected/b9cc6b3b-7847-436f-98e0-83190948a8c5-kube-api-access-qr5pr\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.872082 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.872053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-tmp-dir\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.872082 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.872065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-model-cache\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.872458 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.872123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-home\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.874019 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.873999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-dshm\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.874103 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.874041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cc6b3b-7847-436f-98e0-83190948a8c5-tls-certs\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:16.883268 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:16.883241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr5pr\" (UniqueName: \"kubernetes.io/projected/b9cc6b3b-7847-436f-98e0-83190948a8c5-kube-api-access-qr5pr\") pod \"e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") " pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:17.018160 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:17.018073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:17.170502 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:17.170472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"]
Jul 23 04:17:17.172613 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:17:17.172566    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9cc6b3b_7847_436f_98e0_83190948a8c5.slice/crio-5189e0884e3667ee22ea9385e46309ffc3c3773c2569855c233ec4eb9b1ac619 WatchSource:0}: Error finding container 5189e0884e3667ee22ea9385e46309ffc3c3773c2569855c233ec4eb9b1ac619: Status 404 returned error can't find the container with id 5189e0884e3667ee22ea9385e46309ffc3c3773c2569855c233ec4eb9b1ac619
Jul 23 04:17:18.088903 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:18.088866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls" event={"ID":"b9cc6b3b-7847-436f-98e0-83190948a8c5","Type":"ContainerStarted","Data":"e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc"}
Jul 23 04:17:18.088903 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:18.088911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls" event={"ID":"b9cc6b3b-7847-436f-98e0-83190948a8c5","Type":"ContainerStarted","Data":"5189e0884e3667ee22ea9385e46309ffc3c3773c2569855c233ec4eb9b1ac619"}
Jul 23 04:17:18.107719 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:18.107666    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls" podStartSLOduration=2.107649921 podStartE2EDuration="2.107649921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:17:18.10604277 +0000 UTC m=+1298.825574381" watchObservedRunningTime="2026-07-23 04:17:18.107649921 +0000 UTC m=+1298.827181530"
Jul 23 04:17:19.496536 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:19.496500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:17:23.793175 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.793138    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"]
Jul 23 04:17:23.797144 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.797127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.799703 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.799675    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-819be5df-kserve-self-signed-certs\""
Jul 23 04:17:23.808297 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.808276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"]
Jul 23 04:17:23.833847 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.833815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-dshm\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.833985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.833855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae527df8-f007-494c-8c2f-5660e8c620e7-tls-certs\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.833985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.833945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxgzt\" (UniqueName: \"kubernetes.io/projected/ae527df8-f007-494c-8c2f-5660e8c620e7-kube-api-access-xxgzt\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.833985 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.833980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-tmp-dir\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.834109 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.834030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-home\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.834109 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.834063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-model-cache\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.934746 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.934710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae527df8-f007-494c-8c2f-5660e8c620e7-tls-certs\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.934921 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.934751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxgzt\" (UniqueName: \"kubernetes.io/projected/ae527df8-f007-494c-8c2f-5660e8c620e7-kube-api-access-xxgzt\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.934921 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.934783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-tmp-dir\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.934921 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.934818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-home\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.934921 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.934846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-model-cache\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.935109 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.934961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-dshm\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.935313 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.935280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-tmp-dir\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.935313 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.935298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-home\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.935538 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.935322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-model-cache\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.937244 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.937225    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-dshm\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.937339 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.937322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae527df8-f007-494c-8c2f-5660e8c620e7-tls-certs\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:23.942298 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:23.942280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxgzt\" (UniqueName: \"kubernetes.io/projected/ae527df8-f007-494c-8c2f-5660e8c620e7-kube-api-access-xxgzt\") pod \"e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") " pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:24.106553 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:24.106467    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:24.239382 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:24.239361    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"]
Jul 23 04:17:24.241202 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:17:24.241177    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae527df8_f007_494c_8c2f_5660e8c620e7.slice/crio-8d1cb8c85543e3b0814a6545ac45d7e901d230fc320c002a38cc8d7692522149 WatchSource:0}: Error finding container 8d1cb8c85543e3b0814a6545ac45d7e901d230fc320c002a38cc8d7692522149: Status 404 returned error can't find the container with id 8d1cb8c85543e3b0814a6545ac45d7e901d230fc320c002a38cc8d7692522149
Jul 23 04:17:25.122783 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:25.122750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps" event={"ID":"ae527df8-f007-494c-8c2f-5660e8c620e7","Type":"ContainerStarted","Data":"0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c"}
Jul 23 04:17:25.122783 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:25.122787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps" event={"ID":"ae527df8-f007-494c-8c2f-5660e8c620e7","Type":"ContainerStarted","Data":"8d1cb8c85543e3b0814a6545ac45d7e901d230fc320c002a38cc8d7692522149"}
Jul 23 04:17:25.142301 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:25.142252    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps" podStartSLOduration=2.142236953 podStartE2EDuration="2.142236953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:17:25.139762921 +0000 UTC m=+1305.859294553" watchObservedRunningTime="2026-07-23 04:17:25.142236953 +0000 UTC m=+1305.861768562"
Jul 23 04:17:27.018626 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:27.018543    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:27.018626 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:27.018591    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:27.030556 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:27.030531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:27.143314 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:27.143286    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:34.107380 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:34.107348    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:34.107793 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:34.107432    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:34.119449 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:34.119404    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:34.175804 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:34.175780    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:17:34.985553 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:34.985520    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:17:44.625790 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:44.625758    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"]
Jul 23 04:17:45.390763 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:45.390722    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"]
Jul 23 04:17:49.290584 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:49.290547    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:17:59.647080 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.647030    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls" podUID="b9cc6b3b-7847-436f-98e0-83190948a8c5" containerName="main" containerID="cri-o://e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc" gracePeriod=45
Jul 23 04:17:59.788170 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.788138    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:17:59.901954 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.901901    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:17:59.975801 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.975772    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cc6b3b-7847-436f-98e0-83190948a8c5-tls-certs\") pod \"b9cc6b3b-7847-436f-98e0-83190948a8c5\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") "
Jul 23 04:17:59.975801 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.975805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-home\") pod \"b9cc6b3b-7847-436f-98e0-83190948a8c5\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") "
Jul 23 04:17:59.976033 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.975858    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-dshm\") pod \"b9cc6b3b-7847-436f-98e0-83190948a8c5\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") "
Jul 23 04:17:59.976033 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.975884    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-model-cache\") pod \"b9cc6b3b-7847-436f-98e0-83190948a8c5\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") "
Jul 23 04:17:59.976033 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.975902    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr5pr\" (UniqueName: \"kubernetes.io/projected/b9cc6b3b-7847-436f-98e0-83190948a8c5-kube-api-access-qr5pr\") pod \"b9cc6b3b-7847-436f-98e0-83190948a8c5\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") "
Jul 23 04:17:59.976033 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.975992    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-tmp-dir\") pod \"b9cc6b3b-7847-436f-98e0-83190948a8c5\" (UID: \"b9cc6b3b-7847-436f-98e0-83190948a8c5\") "
Jul 23 04:17:59.976238 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.976134    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9cc6b3b-7847-436f-98e0-83190948a8c5" (UID: "b9cc6b3b-7847-436f-98e0-83190948a8c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:17:59.976238 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.976149    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-home" (OuterVolumeSpecName: "home") pod "b9cc6b3b-7847-436f-98e0-83190948a8c5" (UID: "b9cc6b3b-7847-436f-98e0-83190948a8c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:17:59.976349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.976329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b9cc6b3b-7847-436f-98e0-83190948a8c5" (UID: "b9cc6b3b-7847-436f-98e0-83190948a8c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:17:59.976349 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.976340    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:17:59.976460 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.976360    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:17:59.978202 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.978179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9cc6b3b-7847-436f-98e0-83190948a8c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9cc6b3b-7847-436f-98e0-83190948a8c5" (UID: "b9cc6b3b-7847-436f-98e0-83190948a8c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:17:59.978202 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.978187    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-dshm" (OuterVolumeSpecName: "dshm") pod "b9cc6b3b-7847-436f-98e0-83190948a8c5" (UID: "b9cc6b3b-7847-436f-98e0-83190948a8c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:17:59.978322 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:17:59.978194    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9cc6b3b-7847-436f-98e0-83190948a8c5-kube-api-access-qr5pr" (OuterVolumeSpecName: "kube-api-access-qr5pr") pod "b9cc6b3b-7847-436f-98e0-83190948a8c5" (UID: "b9cc6b3b-7847-436f-98e0-83190948a8c5"). InnerVolumeSpecName "kube-api-access-qr5pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:18:00.077494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.077469    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.077494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.077491    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr5pr\" (UniqueName: \"kubernetes.io/projected/b9cc6b3b-7847-436f-98e0-83190948a8c5-kube-api-access-qr5pr\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.077689 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.077502    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cc6b3b-7847-436f-98e0-83190948a8c5-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.077689 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.077511    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cc6b3b-7847-436f-98e0-83190948a8c5-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.289493 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.289458    2563 generic.go:356] "Generic (PLEG): container finished" podID="b9cc6b3b-7847-436f-98e0-83190948a8c5" containerID="e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc" exitCode=0
Jul 23 04:18:00.289678 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.289508    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"
Jul 23 04:18:00.289678 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.289540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls" event={"ID":"b9cc6b3b-7847-436f-98e0-83190948a8c5","Type":"ContainerDied","Data":"e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc"}
Jul 23 04:18:00.289678 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.289579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls" event={"ID":"b9cc6b3b-7847-436f-98e0-83190948a8c5","Type":"ContainerDied","Data":"5189e0884e3667ee22ea9385e46309ffc3c3773c2569855c233ec4eb9b1ac619"}
Jul 23 04:18:00.289678 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.289597    2563 scope.go:117] "RemoveContainer" containerID="e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc"
Jul 23 04:18:00.298728 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.298712    2563 scope.go:117] "RemoveContainer" containerID="e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc"
Jul 23 04:18:00.299010 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:18:00.298988    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc\": container with ID starting with e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc not found: ID does not exist" containerID="e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc"
Jul 23 04:18:00.299078 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.299024    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc"} err="failed to get container status \"e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc\": rpc error: code = NotFound desc = could not find container \"e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc\": container with ID starting with e0f6f0d5f26bd7a6d2b0cdaed5d4f753c0b0dcda65684972dd05fc9cf0088adc not found: ID does not exist"
Jul 23 04:18:00.309779 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.309754    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"]
Jul 23 04:18:00.313976 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.313953    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-819be5df-kserve-6b7f9cd87-r8lls"]
Jul 23 04:18:00.411852 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.411813    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps" podUID="ae527df8-f007-494c-8c2f-5660e8c620e7" containerName="main" containerID="cri-o://0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c" gracePeriod=45
Jul 23 04:18:00.654166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.654140    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:18:00.685121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxgzt\" (UniqueName: \"kubernetes.io/projected/ae527df8-f007-494c-8c2f-5660e8c620e7-kube-api-access-xxgzt\") pod \"ae527df8-f007-494c-8c2f-5660e8c620e7\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") "
Jul 23 04:18:00.685265 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685127    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-dshm\") pod \"ae527df8-f007-494c-8c2f-5660e8c620e7\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") "
Jul 23 04:18:00.685265 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685169    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-tmp-dir\") pod \"ae527df8-f007-494c-8c2f-5660e8c620e7\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") "
Jul 23 04:18:00.685265 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685220    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-model-cache\") pod \"ae527df8-f007-494c-8c2f-5660e8c620e7\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") "
Jul 23 04:18:00.685265 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae527df8-f007-494c-8c2f-5660e8c620e7-tls-certs\") pod \"ae527df8-f007-494c-8c2f-5660e8c620e7\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") "
Jul 23 04:18:00.685496 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-home\") pod \"ae527df8-f007-494c-8c2f-5660e8c620e7\" (UID: \"ae527df8-f007-494c-8c2f-5660e8c620e7\") "
Jul 23 04:18:00.685552 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "ae527df8-f007-494c-8c2f-5660e8c620e7" (UID: "ae527df8-f007-494c-8c2f-5660e8c620e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:18:00.685614 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685602    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.685714 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685673    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ae527df8-f007-494c-8c2f-5660e8c620e7" (UID: "ae527df8-f007-494c-8c2f-5660e8c620e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:18:00.685714 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.685686    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-home" (OuterVolumeSpecName: "home") pod "ae527df8-f007-494c-8c2f-5660e8c620e7" (UID: "ae527df8-f007-494c-8c2f-5660e8c620e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:18:00.687502 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.687475    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-dshm" (OuterVolumeSpecName: "dshm") pod "ae527df8-f007-494c-8c2f-5660e8c620e7" (UID: "ae527df8-f007-494c-8c2f-5660e8c620e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:18:00.687609 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.687557    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae527df8-f007-494c-8c2f-5660e8c620e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae527df8-f007-494c-8c2f-5660e8c620e7" (UID: "ae527df8-f007-494c-8c2f-5660e8c620e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:18:00.687678 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.687662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae527df8-f007-494c-8c2f-5660e8c620e7-kube-api-access-xxgzt" (OuterVolumeSpecName: "kube-api-access-xxgzt") pod "ae527df8-f007-494c-8c2f-5660e8c620e7" (UID: "ae527df8-f007-494c-8c2f-5660e8c620e7"). InnerVolumeSpecName "kube-api-access-xxgzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:18:00.787094 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.787055    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.787094 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.787084    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.787094 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.787092    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae527df8-f007-494c-8c2f-5660e8c620e7-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.787094 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.787100    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae527df8-f007-494c-8c2f-5660e8c620e7-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:00.787389 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:00.787108    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxgzt\" (UniqueName: \"kubernetes.io/projected/ae527df8-f007-494c-8c2f-5660e8c620e7-kube-api-access-xxgzt\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:18:01.295513 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.295473    2563 generic.go:356] "Generic (PLEG): container finished" podID="ae527df8-f007-494c-8c2f-5660e8c620e7" containerID="0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c" exitCode=0
Jul 23 04:18:01.295688 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.295554    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"
Jul 23 04:18:01.295688 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.295560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps" event={"ID":"ae527df8-f007-494c-8c2f-5660e8c620e7","Type":"ContainerDied","Data":"0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c"}
Jul 23 04:18:01.295688 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.295602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps" event={"ID":"ae527df8-f007-494c-8c2f-5660e8c620e7","Type":"ContainerDied","Data":"8d1cb8c85543e3b0814a6545ac45d7e901d230fc320c002a38cc8d7692522149"}
Jul 23 04:18:01.295688 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.295618    2563 scope.go:117] "RemoveContainer" containerID="0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c"
Jul 23 04:18:01.304611 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.304595    2563 scope.go:117] "RemoveContainer" containerID="0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c"
Jul 23 04:18:01.304858 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:18:01.304832    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c\": container with ID starting with 0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c not found: ID does not exist" containerID="0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c"
Jul 23 04:18:01.304903 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.304867    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c"} err="failed to get container status \"0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c\": rpc error: code = NotFound desc = could not find container \"0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c\": container with ID starting with 0acf9c99d583bf22057bd915c72bcec43b5db487cd6ac45b916e6229e8a3044c not found: ID does not exist"
Jul 23 04:18:01.315789 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.315764    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"]
Jul 23 04:18:01.319655 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.319637    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-819be5df-kserve-8b55c5f59-5f5ps"]
Jul 23 04:18:01.750910 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.750880    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae527df8-f007-494c-8c2f-5660e8c620e7" path="/var/lib/kubelet/pods/ae527df8-f007-494c-8c2f-5660e8c620e7/volumes"
Jul 23 04:18:01.751329 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:01.751313    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9cc6b3b-7847-436f-98e0-83190948a8c5" path="/var/lib/kubelet/pods/b9cc6b3b-7847-436f-98e0-83190948a8c5/volumes"
Jul 23 04:18:06.592218 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:06.592181    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:18:22.086369 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:22.086326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:18:31.485566 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:31.485533    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:18:40.102237 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:40.102161    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:18:52.290573 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:18:52.290538    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:19:01.192427 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:19:01.192374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:19:10.391096 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:19:10.391056    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:19:23.701721 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:19:23.701688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:19:33.391893 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:19:33.391808    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:19:42.592792 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:19:42.592757    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:19:52.095907 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:19:52.095877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:20:05.196930 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:05.196890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:20:14.191558 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:14.191523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:20:26.805521 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:26.805489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:20:37.090257 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:37.090223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:20:39.754179 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:39.754152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:20:39.757399 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:39.757377    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:20:49.605180 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:49.605141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:20:58.495149 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:20:58.495113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:21:20.701696 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:21:20.701661    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:21:35.303369 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:21:35.303335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:22:06.188527 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:22:06.188486    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:22:16.788950 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:22:16.788836    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:22:50.793261 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:22:50.793224    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:23:03.200943 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:23:03.200908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:23:35.106708 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:23:35.106670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:23:47.286567 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:23:47.286530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:24:19.594671 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:24:19.594637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:25:39.784346 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:25:39.784317    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:25:39.788871 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:25:39.788839    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:25:45.501896 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:25:45.501860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:25:52.204727 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:25:52.204689    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:26:03.698451 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:26:03.698396    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:26:13.293489 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:26:13.293454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:26:27.599816 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:26:27.599782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:27:03.697150 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:03.697074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:27:10.008667 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.008633    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"]
Jul 23 04:27:10.009121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.009041    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae527df8-f007-494c-8c2f-5660e8c620e7" containerName="main"
Jul 23 04:27:10.009121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.009052    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae527df8-f007-494c-8c2f-5660e8c620e7" containerName="main"
Jul 23 04:27:10.009121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.009065    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9cc6b3b-7847-436f-98e0-83190948a8c5" containerName="main"
Jul 23 04:27:10.009121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.009070    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9cc6b3b-7847-436f-98e0-83190948a8c5" containerName="main"
Jul 23 04:27:10.009121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.009120    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9cc6b3b-7847-436f-98e0-83190948a8c5" containerName="main"
Jul 23 04:27:10.009443 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.009133    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae527df8-f007-494c-8c2f-5660e8c620e7" containerName="main"
Jul 23 04:27:10.012475 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.012455    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.015355 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.015333    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e0e3ac7b\"/\"e2e-lifecycle-model-e0e3ac7b-kserve-self-signed-certs\""
Jul 23 04:27:10.015505 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.015333    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e0e3ac7b\"/\"openshift-service-ca.crt\""
Jul 23 04:27:10.015505 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.015377    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e0e3ac7b\"/\"kube-root-ca.crt\""
Jul 23 04:27:10.015505 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.015398    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e0e3ac7b\"/\"default-dockercfg-xkflk\""
Jul 23 04:27:10.022850 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.022828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"]
Jul 23 04:27:10.117154 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.117120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-model-cache\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.117323 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.117179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-home\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.117323 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.117207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-dshm\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.117323 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.117230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149ec0ba-0507-4f08-aa97-c4842b32f385-tls-certs\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.117323 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.117314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcjl2\" (UniqueName: \"kubernetes.io/projected/149ec0ba-0507-4f08-aa97-c4842b32f385-kube-api-access-kcjl2\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.117526 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.117342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-tmp-dir\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218547 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-home\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218686 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-dshm\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218686 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218580    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149ec0ba-0507-4f08-aa97-c4842b32f385-tls-certs\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218686 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcjl2\" (UniqueName: \"kubernetes.io/projected/149ec0ba-0507-4f08-aa97-c4842b32f385-kube-api-access-kcjl2\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218686 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-tmp-dir\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218865 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-model-cache\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.218993 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.218965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-home\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.219065 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.219017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-model-cache\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.219120 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.219074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-tmp-dir\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.220901 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.220875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-dshm\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.221156 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.221141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149ec0ba-0507-4f08-aa97-c4842b32f385-tls-certs\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.227260 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.227238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcjl2\" (UniqueName: \"kubernetes.io/projected/149ec0ba-0507-4f08-aa97-c4842b32f385-kube-api-access-kcjl2\") pod \"e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") " pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.323203 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.323128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:10.455636 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.455611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"]
Jul 23 04:27:10.457925 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:27:10.457902    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod149ec0ba_0507_4f08_aa97_c4842b32f385.slice/crio-1a1d90fae2fb7cd9ebd5982fb029b8635e64fb0306495dc468bb1b12546c0d53 WatchSource:0}: Error finding container 1a1d90fae2fb7cd9ebd5982fb029b8635e64fb0306495dc468bb1b12546c0d53: Status 404 returned error can't find the container with id 1a1d90fae2fb7cd9ebd5982fb029b8635e64fb0306495dc468bb1b12546c0d53
Jul 23 04:27:10.931572 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.931536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7" event={"ID":"149ec0ba-0507-4f08-aa97-c4842b32f385","Type":"ContainerStarted","Data":"369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb"}
Jul 23 04:27:10.931770 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.931581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7" event={"ID":"149ec0ba-0507-4f08-aa97-c4842b32f385","Type":"ContainerStarted","Data":"1a1d90fae2fb7cd9ebd5982fb029b8635e64fb0306495dc468bb1b12546c0d53"}
Jul 23 04:27:10.949538 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:10.949491    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7" podStartSLOduration=1.949477093 podStartE2EDuration="1.949477093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:27:10.947038956 +0000 UTC m=+1891.666570566" watchObservedRunningTime="2026-07-23 04:27:10.949477093 +0000 UTC m=+1891.669008702"
Jul 23 04:27:14.805272 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:14.805236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:27:20.323420 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:20.323380    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:20.325902 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:20.323489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:20.336312 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:20.336291    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:20.988530 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:20.988501    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:27:29.697446 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:29.697397    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:27:50.288394 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:27:50.288357    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:28:00.594026 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:00.593982    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:28:03.809921 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:03.809888    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"]
Jul 23 04:28:18.830828 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:18.830779    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7" podUID="149ec0ba-0507-4f08-aa97-c4842b32f385" containerName="main" containerID="cri-o://369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb" gracePeriod=45
Jul 23 04:28:19.079912 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.079891    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:28:19.130302 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-tmp-dir\") pod \"149ec0ba-0507-4f08-aa97-c4842b32f385\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") "
Jul 23 04:28:19.130302 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130230    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149ec0ba-0507-4f08-aa97-c4842b32f385-tls-certs\") pod \"149ec0ba-0507-4f08-aa97-c4842b32f385\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") "
Jul 23 04:28:19.130302 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-home\") pod \"149ec0ba-0507-4f08-aa97-c4842b32f385\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") "
Jul 23 04:28:19.130302 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130299    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcjl2\" (UniqueName: \"kubernetes.io/projected/149ec0ba-0507-4f08-aa97-c4842b32f385-kube-api-access-kcjl2\") pod \"149ec0ba-0507-4f08-aa97-c4842b32f385\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") "
Jul 23 04:28:19.130675 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130323    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-model-cache\") pod \"149ec0ba-0507-4f08-aa97-c4842b32f385\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") "
Jul 23 04:28:19.130675 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-dshm\") pod \"149ec0ba-0507-4f08-aa97-c4842b32f385\" (UID: \"149ec0ba-0507-4f08-aa97-c4842b32f385\") "
Jul 23 04:28:19.130675 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130534    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "149ec0ba-0507-4f08-aa97-c4842b32f385" (UID: "149ec0ba-0507-4f08-aa97-c4842b32f385"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:28:19.130675 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-home" (OuterVolumeSpecName: "home") pod "149ec0ba-0507-4f08-aa97-c4842b32f385" (UID: "149ec0ba-0507-4f08-aa97-c4842b32f385"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:28:19.130675 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130638    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-model-cache" (OuterVolumeSpecName: "model-cache") pod "149ec0ba-0507-4f08-aa97-c4842b32f385" (UID: "149ec0ba-0507-4f08-aa97-c4842b32f385"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:28:19.130929 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130760    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:28:19.130929 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130773    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:28:19.130929 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.130783    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:28:19.132639 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.132611    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-dshm" (OuterVolumeSpecName: "dshm") pod "149ec0ba-0507-4f08-aa97-c4842b32f385" (UID: "149ec0ba-0507-4f08-aa97-c4842b32f385"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:28:19.132776 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.132667    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149ec0ba-0507-4f08-aa97-c4842b32f385-kube-api-access-kcjl2" (OuterVolumeSpecName: "kube-api-access-kcjl2") pod "149ec0ba-0507-4f08-aa97-c4842b32f385" (UID: "149ec0ba-0507-4f08-aa97-c4842b32f385"). InnerVolumeSpecName "kube-api-access-kcjl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:28:19.132834 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.132819    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149ec0ba-0507-4f08-aa97-c4842b32f385-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "149ec0ba-0507-4f08-aa97-c4842b32f385" (UID: "149ec0ba-0507-4f08-aa97-c4842b32f385"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:28:19.231611 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.231582    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149ec0ba-0507-4f08-aa97-c4842b32f385-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:28:19.231611 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.231613    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149ec0ba-0507-4f08-aa97-c4842b32f385-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:28:19.231806 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.231630    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcjl2\" (UniqueName: \"kubernetes.io/projected/149ec0ba-0507-4f08-aa97-c4842b32f385-kube-api-access-kcjl2\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:28:19.271077 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.271041    2563 generic.go:356] "Generic (PLEG): container finished" podID="149ec0ba-0507-4f08-aa97-c4842b32f385" containerID="369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb" exitCode=0
Jul 23 04:28:19.271230 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.271083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7" event={"ID":"149ec0ba-0507-4f08-aa97-c4842b32f385","Type":"ContainerDied","Data":"369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb"}
Jul 23 04:28:19.271230 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.271089    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"
Jul 23 04:28:19.271230 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.271114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7" event={"ID":"149ec0ba-0507-4f08-aa97-c4842b32f385","Type":"ContainerDied","Data":"1a1d90fae2fb7cd9ebd5982fb029b8635e64fb0306495dc468bb1b12546c0d53"}
Jul 23 04:28:19.271230 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.271132    2563 scope.go:117] "RemoveContainer" containerID="369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb"
Jul 23 04:28:19.282712 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.282696    2563 scope.go:117] "RemoveContainer" containerID="369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb"
Jul 23 04:28:19.282992 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:28:19.282969    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb\": container with ID starting with 369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb not found: ID does not exist" containerID="369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb"
Jul 23 04:28:19.283058 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.283000    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb"} err="failed to get container status \"369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb\": rpc error: code = NotFound desc = could not find container \"369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb\": container with ID starting with 369545f5525b9440d887ceb0374a822ec0b9be56ec0af8e6362935094818ecbb not found: ID does not exist"
Jul 23 04:28:19.294294 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.294274    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"]
Jul 23 04:28:19.298399 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.298369    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e0e3ac7b/e2e-lifecycle-model-e0e3ac7b-kserve-7965846f6c-qcqh7"]
Jul 23 04:28:19.750936 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:19.750905    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="149ec0ba-0507-4f08-aa97-c4842b32f385" path="/var/lib/kubelet/pods/149ec0ba-0507-4f08-aa97-c4842b32f385/volumes"
Jul 23 04:28:50.998843 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:28:50.998801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:29:00.591948 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:29:00.591915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:29:51.297664 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:29:51.297626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:30:02.410832 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:02.410755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:30:03.147477 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.147444    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"]
Jul 23 04:30:03.147872 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.147858    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149ec0ba-0507-4f08-aa97-c4842b32f385" containerName="main"
Jul 23 04:30:03.147917 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.147874    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="149ec0ba-0507-4f08-aa97-c4842b32f385" containerName="main"
Jul 23 04:30:03.147959 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.147949    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="149ec0ba-0507-4f08-aa97-c4842b32f385" containerName="main"
Jul 23 04:30:03.151074 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.151058    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.153583 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.153558    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-30964d\"/\"kube-root-ca.crt\""
Jul 23 04:30:03.154105 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.154087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-30964d\"/\"default-dockercfg-mdrq8\""
Jul 23 04:30:03.154105 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.154097    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-30964d\"/\"openshift-service-ca.crt\""
Jul 23 04:30:03.154240 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.154220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-30964d\"/\"test-model-30964d-kserve-self-signed-certs\""
Jul 23 04:30:03.162309 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.162286    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"]
Jul 23 04:30:03.272610 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.272577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-tmp-dir\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.272814 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.272621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-home\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.272814 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.272724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-model-cache\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.272814 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.272773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbnh4\" (UniqueName: \"kubernetes.io/projected/0b920a6d-79b5-4935-a166-971eb81bcd5c-kube-api-access-tbnh4\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.272814 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.272795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-dshm\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.272965 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.272836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b920a6d-79b5-4935-a166-971eb81bcd5c-tls-certs\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.373645 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.373609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-model-cache\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.373808 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.373659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbnh4\" (UniqueName: \"kubernetes.io/projected/0b920a6d-79b5-4935-a166-971eb81bcd5c-kube-api-access-tbnh4\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.373808 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.373685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-dshm\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.373808 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.373724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b920a6d-79b5-4935-a166-971eb81bcd5c-tls-certs\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.373808 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.373754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-tmp-dir\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.373996 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.373890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-home\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.374084 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.374048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-model-cache\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.374213 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.374178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-tmp-dir\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.374213 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.374194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-home\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.376181 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.376159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-dshm\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.376475 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.376458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b920a6d-79b5-4935-a166-971eb81bcd5c-tls-certs\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.383325 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.383304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbnh4\" (UniqueName: \"kubernetes.io/projected/0b920a6d-79b5-4935-a166-971eb81bcd5c-kube-api-access-tbnh4\") pod \"test-model-30964d-kserve-76b8f6d969-knv7t\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") " pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.460543 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.460493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:03.597243 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.597215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"]
Jul 23 04:30:03.599497 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:30:03.599466    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b920a6d_79b5_4935_a166_971eb81bcd5c.slice/crio-93478ea39402b7d8639ab1439fb3df4de92e5804822ef0e0a18e2b50fe91b356 WatchSource:0}: Error finding container 93478ea39402b7d8639ab1439fb3df4de92e5804822ef0e0a18e2b50fe91b356: Status 404 returned error can't find the container with id 93478ea39402b7d8639ab1439fb3df4de92e5804822ef0e0a18e2b50fe91b356
Jul 23 04:30:03.793234 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.793144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t" event={"ID":"0b920a6d-79b5-4935-a166-971eb81bcd5c","Type":"ContainerStarted","Data":"8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f"}
Jul 23 04:30:03.793234 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.793187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t" event={"ID":"0b920a6d-79b5-4935-a166-971eb81bcd5c","Type":"ContainerStarted","Data":"93478ea39402b7d8639ab1439fb3df4de92e5804822ef0e0a18e2b50fe91b356"}
Jul 23 04:30:03.813377 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:03.813324    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t" podStartSLOduration=0.813302904 podStartE2EDuration="813.302904ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:30:03.812972894 +0000 UTC m=+2064.532504519" watchObservedRunningTime="2026-07-23 04:30:03.813302904 +0000 UTC m=+2064.532834515"
Jul 23 04:30:13.461575 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:13.461518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:13.461575 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:13.461588    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:13.473934 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:13.473906    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:13.853823 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:13.853728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:30:18.687960 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:18.687925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:30:27.396396 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.396359    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"]
Jul 23 04:30:27.400150 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.400127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.403159 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.403139    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c7abdc\"/\"kube-root-ca.crt\""
Jul 23 04:30:27.403282 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.403142    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c7abdc\"/\"openshift-service-ca.crt\""
Jul 23 04:30:27.403282 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.403144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c7abdc\"/\"test-model-c7abdc-kserve-self-signed-certs\""
Jul 23 04:30:27.403513 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.403489    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c7abdc\"/\"default-dockercfg-zcks5\""
Jul 23 04:30:27.409337 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.409309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"]
Jul 23 04:30:27.516177 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.516137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tmp-dir\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.516177 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.516175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-model-cache\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.516436 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.516300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-home\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.516436 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.516343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-dshm\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.516436 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.516369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tls-certs\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.516436 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.516393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8cbq\" (UniqueName: \"kubernetes.io/projected/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-kube-api-access-h8cbq\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.616940 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.616895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-home\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.616940 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.616938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-dshm\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617175 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.616960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tls-certs\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617175 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.616985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8cbq\" (UniqueName: \"kubernetes.io/projected/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-kube-api-access-h8cbq\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617175 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.617059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tmp-dir\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617175 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.617079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-model-cache\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617397 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.617372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-home\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617546 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.617526    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tmp-dir\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.617607 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.617564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-model-cache\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.619380 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.619364    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-dshm\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.619748 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.619731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tls-certs\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.628144 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.628120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8cbq\" (UniqueName: \"kubernetes.io/projected/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-kube-api-access-h8cbq\") pod \"test-model-c7abdc-kserve-65bcd8c6bf-wqf9h\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") " pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.710276 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.710246    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:27.846655 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.846508    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"]
Jul 23 04:30:27.848912 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:30:27.848886    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c55a967_1143_4fc4_bfe8_ab92bdd8c637.slice/crio-38cb29ac2683fd2179cc08e9dd0737dfbf02406e89f426a77324536ae0e8dd30 WatchSource:0}: Error finding container 38cb29ac2683fd2179cc08e9dd0737dfbf02406e89f426a77324536ae0e8dd30: Status 404 returned error can't find the container with id 38cb29ac2683fd2179cc08e9dd0737dfbf02406e89f426a77324536ae0e8dd30
Jul 23 04:30:27.912010 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.911975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h" event={"ID":"9c55a967-1143-4fc4-bfe8-ab92bdd8c637","Type":"ContainerStarted","Data":"b2fa996a591a0f4a3d21d12ab4bd06689d38b3d884d45e95fd324cd944abc749"}
Jul 23 04:30:27.912133 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.912016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h" event={"ID":"9c55a967-1143-4fc4-bfe8-ab92bdd8c637","Type":"ContainerStarted","Data":"38cb29ac2683fd2179cc08e9dd0737dfbf02406e89f426a77324536ae0e8dd30"}
Jul 23 04:30:27.931010 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:27.930953    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h" podStartSLOduration=0.930936587 podStartE2EDuration="930.936587ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:30:27.928957031 +0000 UTC m=+2088.648488837" watchObservedRunningTime="2026-07-23 04:30:27.930936587 +0000 UTC m=+2088.650468174"
Jul 23 04:30:34.186304 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:34.186271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:30:37.711056 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:37.711013    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:37.711477 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:37.711090    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:37.723443 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:37.723401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:37.972313 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:37.972238    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:30:39.814447 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:39.814404    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:30:39.819795 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:30:39.819774    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:31:13.104309 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:13.104278    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"]
Jul 23 04:31:14.006551 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:14.006519    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"]
Jul 23 04:31:17.101581 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:17.101535    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:31:28.125142 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.125097    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t" podUID="0b920a6d-79b5-4935-a166-971eb81bcd5c" containerName="main" containerID="cri-o://8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f" gracePeriod=45
Jul 23 04:31:28.382464 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.382390    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:31:28.579942 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.579903    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-model-cache\") pod \"0b920a6d-79b5-4935-a166-971eb81bcd5c\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") "
Jul 23 04:31:28.580124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.579989    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b920a6d-79b5-4935-a166-971eb81bcd5c-tls-certs\") pod \"0b920a6d-79b5-4935-a166-971eb81bcd5c\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") "
Jul 23 04:31:28.580124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbnh4\" (UniqueName: \"kubernetes.io/projected/0b920a6d-79b5-4935-a166-971eb81bcd5c-kube-api-access-tbnh4\") pod \"0b920a6d-79b5-4935-a166-971eb81bcd5c\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") "
Jul 23 04:31:28.580124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-home\") pod \"0b920a6d-79b5-4935-a166-971eb81bcd5c\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") "
Jul 23 04:31:28.580124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-dshm\") pod \"0b920a6d-79b5-4935-a166-971eb81bcd5c\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") "
Jul 23 04:31:28.580302 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580162    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-tmp-dir\") pod \"0b920a6d-79b5-4935-a166-971eb81bcd5c\" (UID: \"0b920a6d-79b5-4935-a166-971eb81bcd5c\") "
Jul 23 04:31:28.580302 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580216    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "0b920a6d-79b5-4935-a166-971eb81bcd5c" (UID: "0b920a6d-79b5-4935-a166-971eb81bcd5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:28.580448 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580328    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-home" (OuterVolumeSpecName: "home") pod "0b920a6d-79b5-4935-a166-971eb81bcd5c" (UID: "0b920a6d-79b5-4935-a166-971eb81bcd5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:28.580545 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0b920a6d-79b5-4935-a166-971eb81bcd5c" (UID: "0b920a6d-79b5-4935-a166-971eb81bcd5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:28.580545 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580531    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:28.580545 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.580548    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:28.582504 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.582483    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b920a6d-79b5-4935-a166-971eb81bcd5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b920a6d-79b5-4935-a166-971eb81bcd5c" (UID: "0b920a6d-79b5-4935-a166-971eb81bcd5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:31:28.582504 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.582482    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-dshm" (OuterVolumeSpecName: "dshm") pod "0b920a6d-79b5-4935-a166-971eb81bcd5c" (UID: "0b920a6d-79b5-4935-a166-971eb81bcd5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:28.582621 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.582532    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b920a6d-79b5-4935-a166-971eb81bcd5c-kube-api-access-tbnh4" (OuterVolumeSpecName: "kube-api-access-tbnh4") pod "0b920a6d-79b5-4935-a166-971eb81bcd5c" (UID: "0b920a6d-79b5-4935-a166-971eb81bcd5c"). InnerVolumeSpecName "kube-api-access-tbnh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:31:28.681267 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.681181    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:28.681267 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.681209    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b920a6d-79b5-4935-a166-971eb81bcd5c-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:28.681267 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.681218    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbnh4\" (UniqueName: \"kubernetes.io/projected/0b920a6d-79b5-4935-a166-971eb81bcd5c-kube-api-access-tbnh4\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:28.681267 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:28.681226    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b920a6d-79b5-4935-a166-971eb81bcd5c-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.027253 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.027212    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h" podUID="9c55a967-1143-4fc4-bfe8-ab92bdd8c637" containerName="main" containerID="cri-o://b2fa996a591a0f4a3d21d12ab4bd06689d38b3d884d45e95fd324cd944abc749" gracePeriod=45
Jul 23 04:31:29.206791 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.206761    2563 generic.go:356] "Generic (PLEG): container finished" podID="0b920a6d-79b5-4935-a166-971eb81bcd5c" containerID="8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f" exitCode=0
Jul 23 04:31:29.207219 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.206822    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"
Jul 23 04:31:29.207219 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.206850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t" event={"ID":"0b920a6d-79b5-4935-a166-971eb81bcd5c","Type":"ContainerDied","Data":"8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f"}
Jul 23 04:31:29.207219 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.206900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t" event={"ID":"0b920a6d-79b5-4935-a166-971eb81bcd5c","Type":"ContainerDied","Data":"93478ea39402b7d8639ab1439fb3df4de92e5804822ef0e0a18e2b50fe91b356"}
Jul 23 04:31:29.207219 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.206924    2563 scope.go:117] "RemoveContainer" containerID="8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f"
Jul 23 04:31:29.209489 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.209460    2563 generic.go:356] "Generic (PLEG): container finished" podID="9c55a967-1143-4fc4-bfe8-ab92bdd8c637" containerID="b2fa996a591a0f4a3d21d12ab4bd06689d38b3d884d45e95fd324cd944abc749" exitCode=0
Jul 23 04:31:29.209605 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.209568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h" event={"ID":"9c55a967-1143-4fc4-bfe8-ab92bdd8c637","Type":"ContainerDied","Data":"b2fa996a591a0f4a3d21d12ab4bd06689d38b3d884d45e95fd324cd944abc749"}
Jul 23 04:31:29.219204 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.219183    2563 scope.go:117] "RemoveContainer" containerID="8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f"
Jul 23 04:31:29.219546 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:31:29.219526    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f\": container with ID starting with 8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f not found: ID does not exist" containerID="8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f"
Jul 23 04:31:29.219623 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.219555    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f"} err="failed to get container status \"8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f\": rpc error: code = NotFound desc = could not find container \"8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f\": container with ID starting with 8a090edf46fcf80ce96a167c267d4aa9385a3673d4da6b272905655ca1fab62f not found: ID does not exist"
Jul 23 04:31:29.233636 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.233610    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"]
Jul 23 04:31:29.235297 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.235276    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-30964d/test-model-30964d-kserve-76b8f6d969-knv7t"]
Jul 23 04:31:29.283904 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.283847    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:31:29.284665 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.284649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-dshm\") pod \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") "
Jul 23 04:31:29.284721 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.284680    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tls-certs\") pod \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") "
Jul 23 04:31:29.284721 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.284708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-model-cache\") pod \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") "
Jul 23 04:31:29.284801 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.284762    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8cbq\" (UniqueName: \"kubernetes.io/projected/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-kube-api-access-h8cbq\") pod \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") "
Jul 23 04:31:29.284838 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.284805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tmp-dir\") pod \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") "
Jul 23 04:31:29.284838 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.284826    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-home\") pod \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\" (UID: \"9c55a967-1143-4fc4-bfe8-ab92bdd8c637\") "
Jul 23 04:31:29.285110 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.285062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c55a967-1143-4fc4-bfe8-ab92bdd8c637" (UID: "9c55a967-1143-4fc4-bfe8-ab92bdd8c637"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:29.285110 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.285093    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c55a967-1143-4fc4-bfe8-ab92bdd8c637" (UID: "9c55a967-1143-4fc4-bfe8-ab92bdd8c637"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:29.285270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.285179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-home" (OuterVolumeSpecName: "home") pod "9c55a967-1143-4fc4-bfe8-ab92bdd8c637" (UID: "9c55a967-1143-4fc4-bfe8-ab92bdd8c637"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:29.285270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.285248    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.285270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.285268    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.287053 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.287027    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c55a967-1143-4fc4-bfe8-ab92bdd8c637" (UID: "9c55a967-1143-4fc4-bfe8-ab92bdd8c637"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:31:29.287053 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.287041    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-dshm" (OuterVolumeSpecName: "dshm") pod "9c55a967-1143-4fc4-bfe8-ab92bdd8c637" (UID: "9c55a967-1143-4fc4-bfe8-ab92bdd8c637"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:31:29.287181 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.287091    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-kube-api-access-h8cbq" (OuterVolumeSpecName: "kube-api-access-h8cbq") pod "9c55a967-1143-4fc4-bfe8-ab92bdd8c637" (UID: "9c55a967-1143-4fc4-bfe8-ab92bdd8c637"). InnerVolumeSpecName "kube-api-access-h8cbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:31:29.386507 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.386472    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.386507 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.386500    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.386507 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.386510    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8cbq\" (UniqueName: \"kubernetes.io/projected/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-kube-api-access-h8cbq\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.386507 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.386519    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c55a967-1143-4fc4-bfe8-ab92bdd8c637-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:31:29.750166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.750133    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b920a6d-79b5-4935-a166-971eb81bcd5c" path="/var/lib/kubelet/pods/0b920a6d-79b5-4935-a166-971eb81bcd5c/volumes"
Jul 23 04:31:29.893438 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:29.893375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:31:30.216824 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:30.216792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h" event={"ID":"9c55a967-1143-4fc4-bfe8-ab92bdd8c637","Type":"ContainerDied","Data":"38cb29ac2683fd2179cc08e9dd0737dfbf02406e89f426a77324536ae0e8dd30"}
Jul 23 04:31:30.216824 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:30.216813    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"
Jul 23 04:31:30.216824 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:30.216834    2563 scope.go:117] "RemoveContainer" containerID="b2fa996a591a0f4a3d21d12ab4bd06689d38b3d884d45e95fd324cd944abc749"
Jul 23 04:31:30.231594 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:30.231567    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"]
Jul 23 04:31:30.241454 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:30.239333    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c7abdc/test-model-c7abdc-kserve-65bcd8c6bf-wqf9h"]
Jul 23 04:31:31.749931 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:31.749895    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c55a967-1143-4fc4-bfe8-ab92bdd8c637" path="/var/lib/kubelet/pods/9c55a967-1143-4fc4-bfe8-ab92bdd8c637/volumes"
Jul 23 04:31:49.416927 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:31:49.416889    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:32:06.898088 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:32:06.898052    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:32:19.101440 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:32:19.101387    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:32:41.294256 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:32:41.294220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:33:11.493521 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:11.493484    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:33:25.799008 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:25.798971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:33:37.002063 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:37.002033    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:33:53.343351 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343317    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"]
Jul 23 04:33:53.343767 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343713    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c55a967-1143-4fc4-bfe8-ab92bdd8c637" containerName="main"
Jul 23 04:33:53.343767 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343725    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c55a967-1143-4fc4-bfe8-ab92bdd8c637" containerName="main"
Jul 23 04:33:53.343767 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343750    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b920a6d-79b5-4935-a166-971eb81bcd5c" containerName="main"
Jul 23 04:33:53.343767 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343756    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b920a6d-79b5-4935-a166-971eb81bcd5c" containerName="main"
Jul 23 04:33:53.343913 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343818    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c55a967-1143-4fc4-bfe8-ab92bdd8c637" containerName="main"
Jul 23 04:33:53.343913 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.343828    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b920a6d-79b5-4935-a166-971eb81bcd5c" containerName="main"
Jul 23 04:33:53.346919 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.346903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.350145 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.350099    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"default-dockercfg-fgrvl\""
Jul 23 04:33:53.350431 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.350395    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"kube-root-ca.crt\""
Jul 23 04:33:53.355796 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.355779    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"openshift-service-ca.crt\""
Jul 23 04:33:53.356024 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.355996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"auth-test-model-5fa859-kserve-self-signed-certs\""
Jul 23 04:33:53.359857 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.359840    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"]
Jul 23 04:33:53.403062 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.403029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-tmp-dir\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.403224 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.403085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-home\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.403224 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.403154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4l4q\" (UniqueName: \"kubernetes.io/projected/81e9b753-f172-474e-80ea-5d83581bcdcf-kube-api-access-p4l4q\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.403224 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.403213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-model-cache\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.403398 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.403299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-dshm\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.403398 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.403331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e9b753-f172-474e-80ea-5d83581bcdcf-tls-certs\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504220 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-model-cache\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-dshm\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e9b753-f172-474e-80ea-5d83581bcdcf-tls-certs\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-tmp-dir\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-home\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504402 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4l4q\" (UniqueName: \"kubernetes.io/projected/81e9b753-f172-474e-80ea-5d83581bcdcf-kube-api-access-p4l4q\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504704 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-model-cache\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504773 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-home\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.504809 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.504754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-tmp-dir\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.506680 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.506653    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-dshm\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.506872 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.506857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e9b753-f172-474e-80ea-5d83581bcdcf-tls-certs\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.525944 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.525917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4l4q\" (UniqueName: \"kubernetes.io/projected/81e9b753-f172-474e-80ea-5d83581bcdcf-kube-api-access-p4l4q\") pod \"auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") " pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.655578 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.655485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:33:53.788879 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.788855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"]
Jul 23 04:33:53.790816 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:33:53.790784    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81e9b753_f172_474e_80ea_5d83581bcdcf.slice/crio-a40baa92a2188c58d1da873b40f15e3da5fdd4ead9b7a2bcd9c66e4c1387409d WatchSource:0}: Error finding container a40baa92a2188c58d1da873b40f15e3da5fdd4ead9b7a2bcd9c66e4c1387409d: Status 404 returned error can't find the container with id a40baa92a2188c58d1da873b40f15e3da5fdd4ead9b7a2bcd9c66e4c1387409d
Jul 23 04:33:53.908694 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.908615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d" event={"ID":"81e9b753-f172-474e-80ea-5d83581bcdcf","Type":"ContainerStarted","Data":"021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b"}
Jul 23 04:33:53.908694 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.908658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d" event={"ID":"81e9b753-f172-474e-80ea-5d83581bcdcf","Type":"ContainerStarted","Data":"a40baa92a2188c58d1da873b40f15e3da5fdd4ead9b7a2bcd9c66e4c1387409d"}
Jul 23 04:33:53.930338 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:53.930280    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d" podStartSLOduration=0.930264841 podStartE2EDuration="930.264841ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:33:53.92861014 +0000 UTC m=+2294.648141751" watchObservedRunningTime="2026-07-23 04:33:53.930264841 +0000 UTC m=+2294.649796450"
Jul 23 04:33:57.088570 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:33:57.088534    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:34:03.655716 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:03.655681    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:34:03.656187 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:03.655737    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:34:03.668057 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:03.668033    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:34:03.966245 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:03.966215    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:34:35.465568 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.465489    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"]
Jul 23 04:34:35.469148 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.469130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.471910 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.471891    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"default-dockercfg-7gqlw\""
Jul 23 04:34:35.472483 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.472466    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"openshift-service-ca.crt\""
Jul 23 04:34:35.473438 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.473397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"kube-root-ca.crt\""
Jul 23 04:34:35.483077 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.483058    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"auth-test-model-ab39e0-kserve-self-signed-certs\""
Jul 23 04:34:35.484940 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.484921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"]
Jul 23 04:34:35.602659 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.602628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-dshm\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.602855 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.602677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tmp-dir\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.602855 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.602747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrd6\" (UniqueName: \"kubernetes.io/projected/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-kube-api-access-cnrd6\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.602855 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.602786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-model-cache\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.602973 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.602861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tls-certs\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.602973 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.602886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-home\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.703828 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.703793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-home\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704017 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.703861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-dshm\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704017 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.703908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tmp-dir\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704017 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.703969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrd6\" (UniqueName: \"kubernetes.io/projected/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-kube-api-access-cnrd6\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704017 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.704011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-model-cache\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704195 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.704094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tls-certs\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704279 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.704259    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-home\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704324 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.704283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tmp-dir\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.704458 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.704441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-model-cache\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.706240 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.706222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-dshm\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.706649 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.706630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tls-certs\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.713474 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.713455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrd6\" (UniqueName: \"kubernetes.io/projected/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-kube-api-access-cnrd6\") pod \"auth-test-model-ab39e0-kserve-5848ddb88c-svrc6\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") " pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.778344 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.778274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:35.913566 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:35.913543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"]
Jul 23 04:34:35.915139 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:34:35.915102    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fa2cd94_a8e5_4bd1_b86e_83499153b55e.slice/crio-f40f40ba5a6a108ea6fdafcf03b439a236c57f78e9028b6bb4ca5758de7dafeb WatchSource:0}: Error finding container f40f40ba5a6a108ea6fdafcf03b439a236c57f78e9028b6bb4ca5758de7dafeb: Status 404 returned error can't find the container with id f40f40ba5a6a108ea6fdafcf03b439a236c57f78e9028b6bb4ca5758de7dafeb
Jul 23 04:34:36.116608 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:36.116523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6" event={"ID":"7fa2cd94-a8e5-4bd1-b86e-83499153b55e","Type":"ContainerStarted","Data":"ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947"}
Jul 23 04:34:36.116608 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:36.116565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6" event={"ID":"7fa2cd94-a8e5-4bd1-b86e-83499153b55e","Type":"ContainerStarted","Data":"f40f40ba5a6a108ea6fdafcf03b439a236c57f78e9028b6bb4ca5758de7dafeb"}
Jul 23 04:34:36.136962 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:36.136919    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6" podStartSLOduration=1.1369052179999999 podStartE2EDuration="1.136905218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:34:36.1348332 +0000 UTC m=+2336.854364812" watchObservedRunningTime="2026-07-23 04:34:36.136905218 +0000 UTC m=+2336.856436827"
Jul 23 04:34:38.793036 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:38.792993    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:34:45.778600 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:45.778563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:45.779067 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:45.778620    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:45.790744 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:45.790719    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:34:46.176994 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:34:46.176911    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:35:09.386713 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:35:09.386633    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:35:26.602273 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:35:26.602235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:35:39.848579 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:35:39.848546    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:35:39.855826 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:35:39.855806    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:35:44.399868 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:35:44.399835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:36:01.289388 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:36:01.289314    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:36:18.789508 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:36:18.789474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:36:36.493659 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:36:36.493616    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:36:47.521316 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:36:47.521279    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"]
Jul 23 04:36:48.302303 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:36:48.302269    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"]
Jul 23 04:36:53.789138 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:36:53.789100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:37:00.203292 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.203257    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"]
Jul 23 04:37:00.206858 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.206839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.209106 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.209088    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"sub-test-model-5fa859-kserve-self-signed-certs\""
Jul 23 04:37:00.219255 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.219232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"]
Jul 23 04:37:00.351988 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.351960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-model-cache\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.352157 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.352015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgmb\" (UniqueName: \"kubernetes.io/projected/abdc3a2c-db85-4121-84d6-f66834e75b36-kube-api-access-wmgmb\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.352157 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.352065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-home\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.352157 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.352081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdc3a2c-db85-4121-84d6-f66834e75b36-tls-certs\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.352157 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.352097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-dshm\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.352157 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.352113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-tmp-dir\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453496 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgmb\" (UniqueName: \"kubernetes.io/projected/abdc3a2c-db85-4121-84d6-f66834e75b36-kube-api-access-wmgmb\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453496 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-home\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453739 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdc3a2c-db85-4121-84d6-f66834e75b36-tls-certs\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453739 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-dshm\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453739 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-tmp-dir\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453739 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-model-cache\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.453962 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.453892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-home\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.454041 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.454013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-model-cache\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.454148 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.454014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-tmp-dir\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.455997 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.455979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-dshm\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.456286 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.456266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdc3a2c-db85-4121-84d6-f66834e75b36-tls-certs\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.461853 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.461830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgmb\" (UniqueName: \"kubernetes.io/projected/abdc3a2c-db85-4121-84d6-f66834e75b36-kube-api-access-wmgmb\") pod \"sub-test-model-5fa859-kserve-7cc5649b75-hlzds\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") " pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.517567 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.517532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:00.652222 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.652196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"]
Jul 23 04:37:00.654347 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:37:00.654322    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabdc3a2c_db85_4121_84d6_f66834e75b36.slice/crio-f838e2d412b62927f812ecb4b15e21e1473a91d02256a32923ec0b35534dc9d6 WatchSource:0}: Error finding container f838e2d412b62927f812ecb4b15e21e1473a91d02256a32923ec0b35534dc9d6: Status 404 returned error can't find the container with id f838e2d412b62927f812ecb4b15e21e1473a91d02256a32923ec0b35534dc9d6
Jul 23 04:37:00.820318 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.820276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds" event={"ID":"abdc3a2c-db85-4121-84d6-f66834e75b36","Type":"ContainerStarted","Data":"04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf"}
Jul 23 04:37:00.820479 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.820325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds" event={"ID":"abdc3a2c-db85-4121-84d6-f66834e75b36","Type":"ContainerStarted","Data":"f838e2d412b62927f812ecb4b15e21e1473a91d02256a32923ec0b35534dc9d6"}
Jul 23 04:37:00.838549 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:00.838505    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds" podStartSLOduration=0.838490278 podStartE2EDuration="838.490278ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:37:00.837524632 +0000 UTC m=+2481.557056243" watchObservedRunningTime="2026-07-23 04:37:00.838490278 +0000 UTC m=+2481.558021939"
Jul 23 04:37:02.541848 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.541808    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d" podUID="81e9b753-f172-474e-80ea-5d83581bcdcf" containerName="main" containerID="cri-o://021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b" gracePeriod=45
Jul 23 04:37:02.798841 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.798782    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:37:02.831823 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.831795    2563 generic.go:356] "Generic (PLEG): container finished" podID="81e9b753-f172-474e-80ea-5d83581bcdcf" containerID="021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b" exitCode=0
Jul 23 04:37:02.831946 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.831836    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"
Jul 23 04:37:02.831946 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.831837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d" event={"ID":"81e9b753-f172-474e-80ea-5d83581bcdcf","Type":"ContainerDied","Data":"021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b"}
Jul 23 04:37:02.832039 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.831943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d" event={"ID":"81e9b753-f172-474e-80ea-5d83581bcdcf","Type":"ContainerDied","Data":"a40baa92a2188c58d1da873b40f15e3da5fdd4ead9b7a2bcd9c66e4c1387409d"}
Jul 23 04:37:02.832039 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.831967    2563 scope.go:117] "RemoveContainer" containerID="021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b"
Jul 23 04:37:02.844128 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.844111    2563 scope.go:117] "RemoveContainer" containerID="021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b"
Jul 23 04:37:02.844382 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:37:02.844363    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b\": container with ID starting with 021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b not found: ID does not exist" containerID="021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b"
Jul 23 04:37:02.844520 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.844390    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b"} err="failed to get container status \"021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b\": rpc error: code = NotFound desc = could not find container \"021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b\": container with ID starting with 021fa9e9e8ca96527bd4ba1ce442aa401f63873bad449405c2cd40b08af9bd4b not found: ID does not exist"
Jul 23 04:37:02.876049 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-tmp-dir\") pod \"81e9b753-f172-474e-80ea-5d83581bcdcf\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") "
Jul 23 04:37:02.876145 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-dshm\") pod \"81e9b753-f172-474e-80ea-5d83581bcdcf\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") "
Jul 23 04:37:02.876145 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4l4q\" (UniqueName: \"kubernetes.io/projected/81e9b753-f172-474e-80ea-5d83581bcdcf-kube-api-access-p4l4q\") pod \"81e9b753-f172-474e-80ea-5d83581bcdcf\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") "
Jul 23 04:37:02.876145 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876116    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-home\") pod \"81e9b753-f172-474e-80ea-5d83581bcdcf\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") "
Jul 23 04:37:02.876288 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-model-cache\") pod \"81e9b753-f172-474e-80ea-5d83581bcdcf\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") "
Jul 23 04:37:02.876288 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e9b753-f172-474e-80ea-5d83581bcdcf-tls-certs\") pod \"81e9b753-f172-474e-80ea-5d83581bcdcf\" (UID: \"81e9b753-f172-474e-80ea-5d83581bcdcf\") "
Jul 23 04:37:02.876388 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876328    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81e9b753-f172-474e-80ea-5d83581bcdcf" (UID: "81e9b753-f172-474e-80ea-5d83581bcdcf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:02.876471 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876435    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-home" (OuterVolumeSpecName: "home") pod "81e9b753-f172-474e-80ea-5d83581bcdcf" (UID: "81e9b753-f172-474e-80ea-5d83581bcdcf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:02.876471 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-model-cache" (OuterVolumeSpecName: "model-cache") pod "81e9b753-f172-474e-80ea-5d83581bcdcf" (UID: "81e9b753-f172-474e-80ea-5d83581bcdcf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:02.876561 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.876488    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:02.878433 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.878385    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81e9b753-f172-474e-80ea-5d83581bcdcf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81e9b753-f172-474e-80ea-5d83581bcdcf" (UID: "81e9b753-f172-474e-80ea-5d83581bcdcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:37:02.878525 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.878471    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e9b753-f172-474e-80ea-5d83581bcdcf-kube-api-access-p4l4q" (OuterVolumeSpecName: "kube-api-access-p4l4q") pod "81e9b753-f172-474e-80ea-5d83581bcdcf" (UID: "81e9b753-f172-474e-80ea-5d83581bcdcf"). InnerVolumeSpecName "kube-api-access-p4l4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:37:02.878572 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.878535    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-dshm" (OuterVolumeSpecName: "dshm") pod "81e9b753-f172-474e-80ea-5d83581bcdcf" (UID: "81e9b753-f172-474e-80ea-5d83581bcdcf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:02.976891 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.976864    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4l4q\" (UniqueName: \"kubernetes.io/projected/81e9b753-f172-474e-80ea-5d83581bcdcf-kube-api-access-p4l4q\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:02.976891 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.976888    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:02.977046 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.976899    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:02.977046 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.976909    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e9b753-f172-474e-80ea-5d83581bcdcf-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:02.977046 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:02.976916    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e9b753-f172-474e-80ea-5d83581bcdcf-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.152070 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.152039    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"]
Jul 23 04:37:03.155871 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.155845    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/auth-test-model-5fa859-kserve-7c84d4fb55-lsb6d"]
Jul 23 04:37:03.293831 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.293796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:37:03.323368 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.323330    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6" podUID="7fa2cd94-a8e5-4bd1-b86e-83499153b55e" containerName="main" containerID="cri-o://ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947" gracePeriod=45
Jul 23 04:37:03.572989 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.572966    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:37:03.581693 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.581670    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-dshm\") pod \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") "
Jul 23 04:37:03.581765 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.581704    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tls-certs\") pod \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") "
Jul 23 04:37:03.581765 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.581753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-model-cache\") pod \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") "
Jul 23 04:37:03.581836 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.581776    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-home\") pod \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") "
Jul 23 04:37:03.581869 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.581852    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tmp-dir\") pod \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") "
Jul 23 04:37:03.581905 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.581892    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnrd6\" (UniqueName: \"kubernetes.io/projected/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-kube-api-access-cnrd6\") pod \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\" (UID: \"7fa2cd94-a8e5-4bd1-b86e-83499153b55e\") "
Jul 23 04:37:03.582086 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.582050    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-model-cache" (OuterVolumeSpecName: "model-cache") pod "7fa2cd94-a8e5-4bd1-b86e-83499153b55e" (UID: "7fa2cd94-a8e5-4bd1-b86e-83499153b55e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:03.582086 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.582077    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-home" (OuterVolumeSpecName: "home") pod "7fa2cd94-a8e5-4bd1-b86e-83499153b55e" (UID: "7fa2cd94-a8e5-4bd1-b86e-83499153b55e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:03.582249 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.582109    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7fa2cd94-a8e5-4bd1-b86e-83499153b55e" (UID: "7fa2cd94-a8e5-4bd1-b86e-83499153b55e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:03.582301 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.582286    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.582365 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.582309    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.582523 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.582464    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.583998 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.583979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-dshm" (OuterVolumeSpecName: "dshm") pod "7fa2cd94-a8e5-4bd1-b86e-83499153b55e" (UID: "7fa2cd94-a8e5-4bd1-b86e-83499153b55e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:37:03.584092 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.584075    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7fa2cd94-a8e5-4bd1-b86e-83499153b55e" (UID: "7fa2cd94-a8e5-4bd1-b86e-83499153b55e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:37:03.584159 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.584137    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-kube-api-access-cnrd6" (OuterVolumeSpecName: "kube-api-access-cnrd6") pod "7fa2cd94-a8e5-4bd1-b86e-83499153b55e" (UID: "7fa2cd94-a8e5-4bd1-b86e-83499153b55e"). InnerVolumeSpecName "kube-api-access-cnrd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:37:03.683310 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.683284    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnrd6\" (UniqueName: \"kubernetes.io/projected/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-kube-api-access-cnrd6\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.683310 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.683309    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.683544 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.683319    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fa2cd94-a8e5-4bd1-b86e-83499153b55e-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:37:03.750460 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.750430    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81e9b753-f172-474e-80ea-5d83581bcdcf" path="/var/lib/kubelet/pods/81e9b753-f172-474e-80ea-5d83581bcdcf/volumes"
Jul 23 04:37:03.837530 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.837501    2563 generic.go:356] "Generic (PLEG): container finished" podID="7fa2cd94-a8e5-4bd1-b86e-83499153b55e" containerID="ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947" exitCode=0
Jul 23 04:37:03.837652 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.837551    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"
Jul 23 04:37:03.837652 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.837583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6" event={"ID":"7fa2cd94-a8e5-4bd1-b86e-83499153b55e","Type":"ContainerDied","Data":"ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947"}
Jul 23 04:37:03.837652 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.837621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6" event={"ID":"7fa2cd94-a8e5-4bd1-b86e-83499153b55e","Type":"ContainerDied","Data":"f40f40ba5a6a108ea6fdafcf03b439a236c57f78e9028b6bb4ca5758de7dafeb"}
Jul 23 04:37:03.837652 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.837645    2563 scope.go:117] "RemoveContainer" containerID="ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947"
Jul 23 04:37:03.846288 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.846271    2563 scope.go:117] "RemoveContainer" containerID="ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947"
Jul 23 04:37:03.846583 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:37:03.846565    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947\": container with ID starting with ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947 not found: ID does not exist" containerID="ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947"
Jul 23 04:37:03.846659 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.846595    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947"} err="failed to get container status \"ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947\": rpc error: code = NotFound desc = could not find container \"ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947\": container with ID starting with ca2fffc0ed766d67bb757edc15d8e0afba1db265bd97447b5084fccb88b5d947 not found: ID does not exist"
Jul 23 04:37:03.856969 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.856948    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"]
Jul 23 04:37:03.861613 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:03.861594    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/auth-test-model-ab39e0-kserve-5848ddb88c-svrc6"]
Jul 23 04:37:05.750694 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:05.750661    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fa2cd94-a8e5-4bd1-b86e-83499153b55e" path="/var/lib/kubelet/pods/7fa2cd94-a8e5-4bd1-b86e-83499153b55e/volumes"
Jul 23 04:37:10.517833 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:10.517799    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:10.518217 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:10.517847    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:10.530346 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:10.530318    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:10.883936 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:10.883852    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:37:40.810542 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.810513    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"]
Jul 23 04:37:40.810916 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.810893    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e9b753-f172-474e-80ea-5d83581bcdcf" containerName="main"
Jul 23 04:37:40.810916 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.810903    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e9b753-f172-474e-80ea-5d83581bcdcf" containerName="main"
Jul 23 04:37:40.810916 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.810912    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fa2cd94-a8e5-4bd1-b86e-83499153b55e" containerName="main"
Jul 23 04:37:40.810916 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.810917    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa2cd94-a8e5-4bd1-b86e-83499153b55e" containerName="main"
Jul 23 04:37:40.811052 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.811002    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81e9b753-f172-474e-80ea-5d83581bcdcf" containerName="main"
Jul 23 04:37:40.811052 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.811011    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fa2cd94-a8e5-4bd1-b86e-83499153b55e" containerName="main"
Jul 23 04:37:40.815486 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.815470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.817852 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.817829    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"kube-root-ca.crt\""
Jul 23 04:37:40.818604 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.818584    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"sub-test-model-ab39e0-kserve-self-signed-certs\""
Jul 23 04:37:40.818686 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.818631    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"default-dockercfg-7gqlw\""
Jul 23 04:37:40.818686 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.818635    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"openshift-service-ca.crt\""
Jul 23 04:37:40.826588 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.826568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"]
Jul 23 04:37:40.833254 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.833226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-home\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.833335 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.833302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-dshm\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.833393 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.833377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tmp-dir\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.833494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.833475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhrts\" (UniqueName: \"kubernetes.io/projected/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-kube-api-access-nhrts\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.833533 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.833513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tls-certs\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.833570 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.833558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-model-cache\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934231 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-home\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934366 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-dshm\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934366 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tmp-dir\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934366 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhrts\" (UniqueName: \"kubernetes.io/projected/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-kube-api-access-nhrts\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934366 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tls-certs\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934593 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-model-cache\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934653 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-home\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934799 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tmp-dir\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.934865 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.934809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-model-cache\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.936694 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.936671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-dshm\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.936995 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.936979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tls-certs\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:40.945156 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:40.945134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhrts\" (UniqueName: \"kubernetes.io/projected/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-kube-api-access-nhrts\") pod \"sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") " pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:41.125186 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:41.125109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:41.256667 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:41.256627    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"]
Jul 23 04:37:41.258527 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:37:41.258501    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e3f458b_e450_4eb3_bc27_6ae20b21b83b.slice/crio-14bf49a5726a8be97f79a5a1e3a4e73ffdc00b1542c332b9c7f43ef3f7c95278 WatchSource:0}: Error finding container 14bf49a5726a8be97f79a5a1e3a4e73ffdc00b1542c332b9c7f43ef3f7c95278: Status 404 returned error can't find the container with id 14bf49a5726a8be97f79a5a1e3a4e73ffdc00b1542c332b9c7f43ef3f7c95278
Jul 23 04:37:42.024530 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:42.024492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8" event={"ID":"4e3f458b-e450-4eb3-bc27-6ae20b21b83b","Type":"ContainerStarted","Data":"3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844"}
Jul 23 04:37:42.024530 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:42.024535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8" event={"ID":"4e3f458b-e450-4eb3-bc27-6ae20b21b83b","Type":"ContainerStarted","Data":"14bf49a5726a8be97f79a5a1e3a4e73ffdc00b1542c332b9c7f43ef3f7c95278"}
Jul 23 04:37:42.044876 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:42.044828    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8" podStartSLOduration=2.04481003 podStartE2EDuration="2.04481003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:37:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:37:42.042375962 +0000 UTC m=+2522.761907564" watchObservedRunningTime="2026-07-23 04:37:42.04481003 +0000 UTC m=+2522.764341641"
Jul 23 04:37:44.715914 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:44.715881    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:37:51.125450 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:51.125389    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:51.125863 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:51.125472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:51.137862 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:51.137834    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:37:52.086634 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:37:52.086607    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:38:03.601259 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:03.601226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:38:17.296307 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:17.296272    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:38:23.544788 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:23.544748    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"]
Jul 23 04:38:24.538280 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:24.538246    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"]
Jul 23 04:38:27.412554 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:27.412523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:38:38.565238 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.565198    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds" podUID="abdc3a2c-db85-4121-84d6-f66834e75b36" containerName="main" containerID="cri-o://04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf" gracePeriod=45
Jul 23 04:38:38.815978 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.815923    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:38:38.860947 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.860916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-model-cache\") pod \"abdc3a2c-db85-4121-84d6-f66834e75b36\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") "
Jul 23 04:38:38.861128 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.860968    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmgmb\" (UniqueName: \"kubernetes.io/projected/abdc3a2c-db85-4121-84d6-f66834e75b36-kube-api-access-wmgmb\") pod \"abdc3a2c-db85-4121-84d6-f66834e75b36\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") "
Jul 23 04:38:38.861128 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.860986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-dshm\") pod \"abdc3a2c-db85-4121-84d6-f66834e75b36\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") "
Jul 23 04:38:38.861128 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.861022    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdc3a2c-db85-4121-84d6-f66834e75b36-tls-certs\") pod \"abdc3a2c-db85-4121-84d6-f66834e75b36\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") "
Jul 23 04:38:38.861301 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.861189    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-model-cache" (OuterVolumeSpecName: "model-cache") pod "abdc3a2c-db85-4121-84d6-f66834e75b36" (UID: "abdc3a2c-db85-4121-84d6-f66834e75b36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:38.861301 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.861215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-tmp-dir\") pod \"abdc3a2c-db85-4121-84d6-f66834e75b36\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") "
Jul 23 04:38:38.861301 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.861251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-home\") pod \"abdc3a2c-db85-4121-84d6-f66834e75b36\" (UID: \"abdc3a2c-db85-4121-84d6-f66834e75b36\") "
Jul 23 04:38:38.861499 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.861443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abdc3a2c-db85-4121-84d6-f66834e75b36" (UID: "abdc3a2c-db85-4121-84d6-f66834e75b36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:38.861552 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.861512    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-home" (OuterVolumeSpecName: "home") pod "abdc3a2c-db85-4121-84d6-f66834e75b36" (UID: "abdc3a2c-db85-4121-84d6-f66834e75b36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:38.862089 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.862064    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:38.862184 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.862092    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:38.862184 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.862106    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:38.863369 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.863346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-dshm" (OuterVolumeSpecName: "dshm") pod "abdc3a2c-db85-4121-84d6-f66834e75b36" (UID: "abdc3a2c-db85-4121-84d6-f66834e75b36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:38.863494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.863376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abdc3a2c-db85-4121-84d6-f66834e75b36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abdc3a2c-db85-4121-84d6-f66834e75b36" (UID: "abdc3a2c-db85-4121-84d6-f66834e75b36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:38:38.863494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.863438    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdc3a2c-db85-4121-84d6-f66834e75b36-kube-api-access-wmgmb" (OuterVolumeSpecName: "kube-api-access-wmgmb") pod "abdc3a2c-db85-4121-84d6-f66834e75b36" (UID: "abdc3a2c-db85-4121-84d6-f66834e75b36"). InnerVolumeSpecName "kube-api-access-wmgmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:38:38.962784 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.962755    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmgmb\" (UniqueName: \"kubernetes.io/projected/abdc3a2c-db85-4121-84d6-f66834e75b36-kube-api-access-wmgmb\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:38.962784 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.962779    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abdc3a2c-db85-4121-84d6-f66834e75b36-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:38.962784 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:38.962788    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdc3a2c-db85-4121-84d6-f66834e75b36-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:39.303858 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.303824    2563 generic.go:356] "Generic (PLEG): container finished" podID="abdc3a2c-db85-4121-84d6-f66834e75b36" containerID="04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf" exitCode=0
Jul 23 04:38:39.304044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.303869    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"
Jul 23 04:38:39.304044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.303919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds" event={"ID":"abdc3a2c-db85-4121-84d6-f66834e75b36","Type":"ContainerDied","Data":"04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf"}
Jul 23 04:38:39.304044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.303967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds" event={"ID":"abdc3a2c-db85-4121-84d6-f66834e75b36","Type":"ContainerDied","Data":"f838e2d412b62927f812ecb4b15e21e1473a91d02256a32923ec0b35534dc9d6"}
Jul 23 04:38:39.304044 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.303990    2563 scope.go:117] "RemoveContainer" containerID="04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf"
Jul 23 04:38:39.313156 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.313139    2563 scope.go:117] "RemoveContainer" containerID="04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf"
Jul 23 04:38:39.313450 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:38:39.313430    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf\": container with ID starting with 04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf not found: ID does not exist" containerID="04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf"
Jul 23 04:38:39.313502 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.313460    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf"} err="failed to get container status \"04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf\": rpc error: code = NotFound desc = could not find container \"04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf\": container with ID starting with 04ddeb54aa0905b8668cf653eb7460fb5761a8dce48e03532842d8c890a7aecf not found: ID does not exist"
Jul 23 04:38:39.325350 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.325322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"]
Jul 23 04:38:39.329330 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.329306    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/sub-test-model-5fa859-kserve-7cc5649b75-hlzds"]
Jul 23 04:38:39.558501 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.558375    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8" podUID="4e3f458b-e450-4eb3-bc27-6ae20b21b83b" containerName="main" containerID="cri-o://3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844" gracePeriod=45
Jul 23 04:38:39.753304 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.753271    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdc3a2c-db85-4121-84d6-f66834e75b36" path="/var/lib/kubelet/pods/abdc3a2c-db85-4121-84d6-f66834e75b36/volumes"
Jul 23 04:38:39.801193 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.801174    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:38:39.869121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-home\") pod \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") "
Jul 23 04:38:39.869121 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869106    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tmp-dir\") pod \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") "
Jul 23 04:38:39.869320 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tls-certs\") pod \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") "
Jul 23 04:38:39.869320 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-dshm\") pod \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") "
Jul 23 04:38:39.869320 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-model-cache\") pod \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") "
Jul 23 04:38:39.869320 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869292    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhrts\" (UniqueName: \"kubernetes.io/projected/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-kube-api-access-nhrts\") pod \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\" (UID: \"4e3f458b-e450-4eb3-bc27-6ae20b21b83b\") "
Jul 23 04:38:39.869566 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869345    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-home" (OuterVolumeSpecName: "home") pod "4e3f458b-e450-4eb3-bc27-6ae20b21b83b" (UID: "4e3f458b-e450-4eb3-bc27-6ae20b21b83b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:39.869566 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869455    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e3f458b-e450-4eb3-bc27-6ae20b21b83b" (UID: "4e3f458b-e450-4eb3-bc27-6ae20b21b83b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:39.869979 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e3f458b-e450-4eb3-bc27-6ae20b21b83b" (UID: "4e3f458b-e450-4eb3-bc27-6ae20b21b83b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:39.869979 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869743    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:39.869979 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869764    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:39.869979 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.869791    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:39.871482 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.871457    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e3f458b-e450-4eb3-bc27-6ae20b21b83b" (UID: "4e3f458b-e450-4eb3-bc27-6ae20b21b83b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:38:39.871482 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.871473    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-dshm" (OuterVolumeSpecName: "dshm") pod "4e3f458b-e450-4eb3-bc27-6ae20b21b83b" (UID: "4e3f458b-e450-4eb3-bc27-6ae20b21b83b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:38:39.871629 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.871613    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-kube-api-access-nhrts" (OuterVolumeSpecName: "kube-api-access-nhrts") pod "4e3f458b-e450-4eb3-bc27-6ae20b21b83b" (UID: "4e3f458b-e450-4eb3-bc27-6ae20b21b83b"). InnerVolumeSpecName "kube-api-access-nhrts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:38:39.970900 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.970875    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:39.970900 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.970899    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:39.970900 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:39.970908    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhrts\" (UniqueName: \"kubernetes.io/projected/4e3f458b-e450-4eb3-bc27-6ae20b21b83b-kube-api-access-nhrts\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:38:40.310026 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.309992    2563 generic.go:356] "Generic (PLEG): container finished" podID="4e3f458b-e450-4eb3-bc27-6ae20b21b83b" containerID="3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844" exitCode=0
Jul 23 04:38:40.310204 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.310045    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"
Jul 23 04:38:40.310204 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.310041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8" event={"ID":"4e3f458b-e450-4eb3-bc27-6ae20b21b83b","Type":"ContainerDied","Data":"3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844"}
Jul 23 04:38:40.310204 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.310165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8" event={"ID":"4e3f458b-e450-4eb3-bc27-6ae20b21b83b","Type":"ContainerDied","Data":"14bf49a5726a8be97f79a5a1e3a4e73ffdc00b1542c332b9c7f43ef3f7c95278"}
Jul 23 04:38:40.310204 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.310189    2563 scope.go:117] "RemoveContainer" containerID="3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844"
Jul 23 04:38:40.319520 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.319503    2563 scope.go:117] "RemoveContainer" containerID="3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844"
Jul 23 04:38:40.319759 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:38:40.319737    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844\": container with ID starting with 3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844 not found: ID does not exist" containerID="3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844"
Jul 23 04:38:40.319801 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.319767    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844"} err="failed to get container status \"3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844\": rpc error: code = NotFound desc = could not find container \"3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844\": container with ID starting with 3de578965897cdf6c0ed3a2200999aeadf5c33c199f867619e6d566a8e9e4844 not found: ID does not exist"
Jul 23 04:38:40.332208 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.332182    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"]
Jul 23 04:38:40.335570 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:40.335549    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/sub-test-model-ab39e0-kserve-85654bc6cf-p5qw8"]
Jul 23 04:38:41.751083 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:41.751052    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e3f458b-e450-4eb3-bc27-6ae20b21b83b" path="/var/lib/kubelet/pods/4e3f458b-e450-4eb3-bc27-6ae20b21b83b/volumes"
Jul 23 04:38:44.093186 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:44.093150    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:38:53.412278 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412243    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"]
Jul 23 04:38:53.412777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412668    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdc3a2c-db85-4121-84d6-f66834e75b36" containerName="main"
Jul 23 04:38:53.412777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412680    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdc3a2c-db85-4121-84d6-f66834e75b36" containerName="main"
Jul 23 04:38:53.412777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412692    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3f458b-e450-4eb3-bc27-6ae20b21b83b" containerName="main"
Jul 23 04:38:53.412777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412698    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3f458b-e450-4eb3-bc27-6ae20b21b83b" containerName="main"
Jul 23 04:38:53.412777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412771    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3f458b-e450-4eb3-bc27-6ae20b21b83b" containerName="main"
Jul 23 04:38:53.412777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.412780    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdc3a2c-db85-4121-84d6-f66834e75b36" containerName="main"
Jul 23 04:38:53.416221 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.416203    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.418579 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.418558    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"openshift-service-ca.crt\""
Jul 23 04:38:53.418695 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.418675    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"kube-root-ca.crt\""
Jul 23 04:38:53.419371 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.419356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"default-dockercfg-7gqlw\""
Jul 23 04:38:53.419959 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.419943    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab39e0\"/\"rate-test-model-ab39e0-kserve-self-signed-certs\""
Jul 23 04:38:53.429178 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.429156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"]
Jul 23 04:38:53.481505 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.481480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-home\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.481619 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.481526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rvdt\" (UniqueName: \"kubernetes.io/projected/743faadd-1937-4d2e-870a-d2ed8f47af54-kube-api-access-9rvdt\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.481619 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.481562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-model-cache\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.481619 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.481608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/743faadd-1937-4d2e-870a-d2ed8f47af54-tls-certs\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.481777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.481646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-dshm\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.481777 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.481676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-tmp-dir\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.582864 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.582837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rvdt\" (UniqueName: \"kubernetes.io/projected/743faadd-1937-4d2e-870a-d2ed8f47af54-kube-api-access-9rvdt\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.582974 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.582873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-model-cache\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583032 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/743faadd-1937-4d2e-870a-d2ed8f47af54-tls-certs\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583100 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-dshm\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583174 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-tmp-dir\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583299 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-model-cache\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583299 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-home\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583503 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-tmp-dir\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.583566 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.583553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-home\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.585366 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.585348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-dshm\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.585666 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.585648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/743faadd-1937-4d2e-870a-d2ed8f47af54-tls-certs\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.590543 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.590523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rvdt\" (UniqueName: \"kubernetes.io/projected/743faadd-1937-4d2e-870a-d2ed8f47af54-kube-api-access-9rvdt\") pod \"rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") " pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.725403 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.725375    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:38:53.863484 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:53.863448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"]
Jul 23 04:38:53.866558 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:38:53.866520    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod743faadd_1937_4d2e_870a_d2ed8f47af54.slice/crio-8f865a6c9b21995daed796e3f776b4f4ccda59b54fbe7bda5eed4a1dbb48c993 WatchSource:0}: Error finding container 8f865a6c9b21995daed796e3f776b4f4ccda59b54fbe7bda5eed4a1dbb48c993: Status 404 returned error can't find the container with id 8f865a6c9b21995daed796e3f776b4f4ccda59b54fbe7bda5eed4a1dbb48c993
Jul 23 04:38:54.381110 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:54.381076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf" event={"ID":"743faadd-1937-4d2e-870a-d2ed8f47af54","Type":"ContainerStarted","Data":"d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743"}
Jul 23 04:38:54.381296 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:54.381116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf" event={"ID":"743faadd-1937-4d2e-870a-d2ed8f47af54","Type":"ContainerStarted","Data":"8f865a6c9b21995daed796e3f776b4f4ccda59b54fbe7bda5eed4a1dbb48c993"}
Jul 23 04:38:54.400291 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:54.400240    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf" podStartSLOduration=1.400225606 podStartE2EDuration="1.400225606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:38:54.398869558 +0000 UTC m=+2595.118401171" watchObservedRunningTime="2026-07-23 04:38:54.400225606 +0000 UTC m=+2595.119757217"
Jul 23 04:38:56.695305 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:38:56.695270    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:39:03.725564 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:03.725475    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:39:03.725564 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:03.725532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:39:03.737929 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:03.737902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:39:04.435650 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:04.435621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:39:11.795497 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:11.795467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:39:26.496961 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:26.496924    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:39:34.311584 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:34.311553    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"]
Jul 23 04:39:41.487282 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:41.487243    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:39:49.332202 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.332137    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf" podUID="743faadd-1937-4d2e-870a-d2ed8f47af54" containerName="main" containerID="cri-o://d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743" gracePeriod=45
Jul 23 04:39:49.579626 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.579603    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:39:49.652465 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.652371    2563 generic.go:356] "Generic (PLEG): container finished" podID="743faadd-1937-4d2e-870a-d2ed8f47af54" containerID="d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743" exitCode=0
Jul 23 04:39:49.652465 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.652450    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"
Jul 23 04:39:49.652639 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.652468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf" event={"ID":"743faadd-1937-4d2e-870a-d2ed8f47af54","Type":"ContainerDied","Data":"d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743"}
Jul 23 04:39:49.652639 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.652515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf" event={"ID":"743faadd-1937-4d2e-870a-d2ed8f47af54","Type":"ContainerDied","Data":"8f865a6c9b21995daed796e3f776b4f4ccda59b54fbe7bda5eed4a1dbb48c993"}
Jul 23 04:39:49.652639 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.652537    2563 scope.go:117] "RemoveContainer" containerID="d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743"
Jul 23 04:39:49.660927 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.660912    2563 scope.go:117] "RemoveContainer" containerID="d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743"
Jul 23 04:39:49.661193 ip-10-0-131-0 kubenswrapper[2563]: E0723 04:39:49.661173    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743\": container with ID starting with d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743 not found: ID does not exist" containerID="d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743"
Jul 23 04:39:49.661255 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.661200    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743"} err="failed to get container status \"d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743\": rpc error: code = NotFound desc = could not find container \"d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743\": container with ID starting with d6f2bda1fcfb47ed5a6c258762ade72e5b63dfb79ac5b224f0dd580dee4c0743 not found: ID does not exist"
Jul 23 04:39:49.691069 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691045    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-dshm\") pod \"743faadd-1937-4d2e-870a-d2ed8f47af54\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") "
Jul 23 04:39:49.691164 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691094    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rvdt\" (UniqueName: \"kubernetes.io/projected/743faadd-1937-4d2e-870a-d2ed8f47af54-kube-api-access-9rvdt\") pod \"743faadd-1937-4d2e-870a-d2ed8f47af54\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") "
Jul 23 04:39:49.691164 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-tmp-dir\") pod \"743faadd-1937-4d2e-870a-d2ed8f47af54\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") "
Jul 23 04:39:49.691164 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691145    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-home\") pod \"743faadd-1937-4d2e-870a-d2ed8f47af54\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") "
Jul 23 04:39:49.691291 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/743faadd-1937-4d2e-870a-d2ed8f47af54-tls-certs\") pod \"743faadd-1937-4d2e-870a-d2ed8f47af54\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") "
Jul 23 04:39:49.691291 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691246    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-model-cache\") pod \"743faadd-1937-4d2e-870a-d2ed8f47af54\" (UID: \"743faadd-1937-4d2e-870a-d2ed8f47af54\") "
Jul 23 04:39:49.691494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-home" (OuterVolumeSpecName: "home") pod "743faadd-1937-4d2e-870a-d2ed8f47af54" (UID: "743faadd-1937-4d2e-870a-d2ed8f47af54"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:49.691494 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691476    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "743faadd-1937-4d2e-870a-d2ed8f47af54" (UID: "743faadd-1937-4d2e-870a-d2ed8f47af54"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:49.691650 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691622    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-model-cache" (OuterVolumeSpecName: "model-cache") pod "743faadd-1937-4d2e-870a-d2ed8f47af54" (UID: "743faadd-1937-4d2e-870a-d2ed8f47af54"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:49.691726 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691665    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-tmp-dir\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.691862 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.691800    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-home\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.693312 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.693280    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-dshm" (OuterVolumeSpecName: "dshm") pod "743faadd-1937-4d2e-870a-d2ed8f47af54" (UID: "743faadd-1937-4d2e-870a-d2ed8f47af54"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:49.693312 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.693295    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/743faadd-1937-4d2e-870a-d2ed8f47af54-kube-api-access-9rvdt" (OuterVolumeSpecName: "kube-api-access-9rvdt") pod "743faadd-1937-4d2e-870a-d2ed8f47af54" (UID: "743faadd-1937-4d2e-870a-d2ed8f47af54"). InnerVolumeSpecName "kube-api-access-9rvdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:39:49.693457 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.693307    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743faadd-1937-4d2e-870a-d2ed8f47af54-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "743faadd-1937-4d2e-870a-d2ed8f47af54" (UID: "743faadd-1937-4d2e-870a-d2ed8f47af54"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:39:49.792956 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.792925    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rvdt\" (UniqueName: \"kubernetes.io/projected/743faadd-1937-4d2e-870a-d2ed8f47af54-kube-api-access-9rvdt\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.792956 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.792952    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/743faadd-1937-4d2e-870a-d2ed8f47af54-tls-certs\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.793111 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.792966    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-model-cache\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.793111 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.792977    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/743faadd-1937-4d2e-870a-d2ed8f47af54-dshm\") on node \"ip-10-0-131-0.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.968389 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.968358    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"]
Jul 23 04:39:49.972727 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:49.972696    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ab39e0/rate-test-model-ab39e0-kserve-7f5c78cdc5-9t6pf"]
Jul 23 04:39:51.750423 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:39:51.750376    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="743faadd-1937-4d2e-870a-d2ed8f47af54" path="/var/lib/kubelet/pods/743faadd-1937-4d2e-870a-d2ed8f47af54/volumes"
Jul 23 04:40:06.509665 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:06.509628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:40:29.289629 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:29.289597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l884s"]
Jul 23 04:40:39.888648 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:39.888577    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:40:39.895096 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:39.895075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9qstf_032d1254-cfe2-47d6-ba4d-80752c4f2ae4/ovn-acl-logging/0.log"
Jul 23 04:40:44.828432 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:44.828387    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-4ktqs_841f5b8d-7249-4352-bffa-6c27f35e34d6/copy-manifests/0.log"
Jul 23 04:40:44.842708 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:44.842681    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-4ktqs_841f5b8d-7249-4352-bffa-6c27f35e34d6/manager/0.log"
Jul 23 04:40:47.520071 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:47.520035    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mxm69_5771e8e3-ebd4-4ef8-87a5-dd360afc50e8/kuadrant-console-plugin/0.log"
Jul 23 04:40:47.898774 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:47.898688    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-l884s_b633855a-5919-405e-9866-16a6d49e5c4d/limitador/0.log"
Jul 23 04:40:48.638565 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:48.638534    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-4zmtr_6bb86030-2683-4b17-bd80-d07ed1845dfd/kube-auth-proxy/0.log"
Jul 23 04:40:49.726760 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:49.726729    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-q5b6w_8cf01197-ef21-405c-8870-75057f3ecaef/main/0.log"
Jul 23 04:40:49.844869 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:49.844845    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5cw2s_6242d091-0c2f-4de6-9dc3-c1148e2d236c/main/0.log"
Jul 23 04:40:50.072574 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:50.072495    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-d8v84_adb0b845-04d6-445e-ac9c-262bed6ca08c/main/0.log"
Jul 23 04:40:58.439924 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:58.439892    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z9cwq_4052be7b-0b50-40e8-a250-3e251a80d7d9/global-pull-secret-syncer/0.log"
Jul 23 04:40:58.562164 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:58.562137    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rcdxk_b3e165c3-a86b-4cd8-808f-67726369adb2/konnectivity-agent/0.log"
Jul 23 04:40:58.593190 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:40:58.593162    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-0.ec2.internal_df25c8e9f4a25a3267946e74f3761d4c/haproxy/0.log"
Jul 23 04:41:03.830278 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:03.830250    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mxm69_5771e8e3-ebd4-4ef8-87a5-dd360afc50e8/kuadrant-console-plugin/0.log"
Jul 23 04:41:04.067373 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:04.067336    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-l884s_b633855a-5919-405e-9866-16a6d49e5c4d/limitador/0.log"
Jul 23 04:41:05.903856 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:05.903829    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/alertmanager/0.log"
Jul 23 04:41:05.931569 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:05.931543    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/config-reloader/0.log"
Jul 23 04:41:05.957806 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:05.957785    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/kube-rbac-proxy-web/0.log"
Jul 23 04:41:05.991583 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:05.991547    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/kube-rbac-proxy/0.log"
Jul 23 04:41:06.020534 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.020511    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/kube-rbac-proxy-metric/0.log"
Jul 23 04:41:06.052946 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.052921    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/prom-label-proxy/0.log"
Jul 23 04:41:06.082516 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.082495    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcb81eff-4cca-4e3d-88ca-53608d2cf5ce/init-config-reloader/0.log"
Jul 23 04:41:06.161390 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.161312    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-x6fqk_c4da9a14-ffa8-4545-8826-343b63133c1f/kube-state-metrics/0.log"
Jul 23 04:41:06.191884 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.191854    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-x6fqk_c4da9a14-ffa8-4545-8826-343b63133c1f/kube-rbac-proxy-main/0.log"
Jul 23 04:41:06.220642 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.220618    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-x6fqk_c4da9a14-ffa8-4545-8826-343b63133c1f/kube-rbac-proxy-self/0.log"
Jul 23 04:41:06.257356 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.257335    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-867b956d86-md2s6_44408668-1169-4463-a5f3-2721bf8da747/metrics-server/0.log"
Jul 23 04:41:06.288433 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.288385    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-xkvrq_6f2ad05c-2bd6-45f9-8c44-8b4a3b839606/monitoring-plugin/0.log"
Jul 23 04:41:06.328380 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.328356    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gz756_a12afeee-5893-4a93-83d1-f0ada3404c9e/node-exporter/0.log"
Jul 23 04:41:06.404334 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.404304    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gz756_a12afeee-5893-4a93-83d1-f0ada3404c9e/kube-rbac-proxy/0.log"
Jul 23 04:41:06.444246 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.444221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gz756_a12afeee-5893-4a93-83d1-f0ada3404c9e/init-textfile/0.log"
Jul 23 04:41:06.726965 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.726892    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vvzhp_3f726308-3e99-4682-933f-13a03681d96a/kube-rbac-proxy-main/0.log"
Jul 23 04:41:06.757449 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.757426    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vvzhp_3f726308-3e99-4682-933f-13a03681d96a/kube-rbac-proxy-self/0.log"
Jul 23 04:41:06.789682 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.789655    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vvzhp_3f726308-3e99-4682-933f-13a03681d96a/openshift-state-metrics/0.log"
Jul 23 04:41:06.843837 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.843811    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/prometheus/0.log"
Jul 23 04:41:06.874532 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.874495    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/config-reloader/0.log"
Jul 23 04:41:06.904353 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.904330    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/thanos-sidecar/0.log"
Jul 23 04:41:06.931976 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.931956    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/kube-rbac-proxy-web/0.log"
Jul 23 04:41:06.958110 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.958084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/kube-rbac-proxy/0.log"
Jul 23 04:41:06.984779 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:06.984714    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/kube-rbac-proxy-thanos/0.log"
Jul 23 04:41:07.008834 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.008813    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c245edbf-6e31-4c49-9523-01d215b62539/init-config-reloader/0.log"
Jul 23 04:41:07.052925 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.052901    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-qfb5s_0f0ea308-d3c2-4745-aff8-cd7765d594aa/prometheus-operator/0.log"
Jul 23 04:41:07.074908 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.074888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-qfb5s_0f0ea308-d3c2-4745-aff8-cd7765d594aa/kube-rbac-proxy/0.log"
Jul 23 04:41:07.101030 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.101008    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-25bvg_901bfc22-c9e2-4e5d-b0bb-603cb53daa45/prometheus-operator-admission-webhook/0.log"
Jul 23 04:41:07.138445 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.138400    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b84dcb675-lm529_14ddbcfc-db2b-433d-a8a0-f43adce481de/telemeter-client/0.log"
Jul 23 04:41:07.171498 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.171475    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b84dcb675-lm529_14ddbcfc-db2b-433d-a8a0-f43adce481de/reload/0.log"
Jul 23 04:41:07.197472 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.197451    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b84dcb675-lm529_14ddbcfc-db2b-433d-a8a0-f43adce481de/kube-rbac-proxy/0.log"
Jul 23 04:41:07.237681 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.237627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-744786bd49-t9xps_4bc3eb25-192a-4840-89b8-912db55cb57b/thanos-query/0.log"
Jul 23 04:41:07.259587 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.259570    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-744786bd49-t9xps_4bc3eb25-192a-4840-89b8-912db55cb57b/kube-rbac-proxy-web/0.log"
Jul 23 04:41:07.281353 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.281337    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-744786bd49-t9xps_4bc3eb25-192a-4840-89b8-912db55cb57b/kube-rbac-proxy/0.log"
Jul 23 04:41:07.310653 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.310627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-744786bd49-t9xps_4bc3eb25-192a-4840-89b8-912db55cb57b/prom-label-proxy/0.log"
Jul 23 04:41:07.335343 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.335320    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-744786bd49-t9xps_4bc3eb25-192a-4840-89b8-912db55cb57b/kube-rbac-proxy-rules/0.log"
Jul 23 04:41:07.355795 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:07.355777    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-744786bd49-t9xps_4bc3eb25-192a-4840-89b8-912db55cb57b/kube-rbac-proxy-metrics/0.log"
Jul 23 04:41:08.467951 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.467922    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"]
Jul 23 04:41:08.468312 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.468297    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="743faadd-1937-4d2e-870a-d2ed8f47af54" containerName="main"
Jul 23 04:41:08.468312 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.468307    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="743faadd-1937-4d2e-870a-d2ed8f47af54" containerName="main"
Jul 23 04:41:08.468396 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.468370    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="743faadd-1937-4d2e-870a-d2ed8f47af54" containerName="main"
Jul 23 04:41:08.471817 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.471799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.474033 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.474011    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-czc5s\"/\"default-dockercfg-qqcf2\""
Jul 23 04:41:08.474166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.474069    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czc5s\"/\"kube-root-ca.crt\""
Jul 23 04:41:08.474166 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.474068    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czc5s\"/\"openshift-service-ca.crt\""
Jul 23 04:41:08.481601 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.481555    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"]
Jul 23 04:41:08.595919 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.595891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkjn6\" (UniqueName: \"kubernetes.io/projected/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-kube-api-access-rkjn6\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.595919 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.595920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-proc\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.596101 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.595987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-podres\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.596101 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.596033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-lib-modules\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.596101 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.596073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-sys\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.696967 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.696938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkjn6\" (UniqueName: \"kubernetes.io/projected/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-kube-api-access-rkjn6\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.696983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-proc\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-podres\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-lib-modules\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697118    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-proc\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-sys\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-sys\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-podres\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.697270 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.697266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-lib-modules\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.705234 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.705216    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkjn6\" (UniqueName: \"kubernetes.io/projected/8ab65753-f5b7-49aa-a2d7-2fdeadcac466-kube-api-access-rkjn6\") pod \"perf-node-gather-daemonset-nkwfh\" (UID: \"8ab65753-f5b7-49aa-a2d7-2fdeadcac466\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.782778 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.782712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:08.915708 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:08.915685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"]
Jul 23 04:41:08.918099 ip-10-0-131-0 kubenswrapper[2563]: W0723 04:41:08.918076    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8ab65753_f5b7_49aa_a2d7_2fdeadcac466.slice/crio-4ee23405a3672d03676de2b0280566f8b29b7a44d4b31abe2be7b347d71f2a78 WatchSource:0}: Error finding container 4ee23405a3672d03676de2b0280566f8b29b7a44d4b31abe2be7b347d71f2a78: Status 404 returned error can't find the container with id 4ee23405a3672d03676de2b0280566f8b29b7a44d4b31abe2be7b347d71f2a78
Jul 23 04:41:09.028598 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:09.028568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh" event={"ID":"8ab65753-f5b7-49aa-a2d7-2fdeadcac466","Type":"ContainerStarted","Data":"109b5c8bbecafcb82ad4881fb142e935e695216111f72c4533e46519f2ff88e3"}
Jul 23 04:41:09.028791 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:09.028625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh" event={"ID":"8ab65753-f5b7-49aa-a2d7-2fdeadcac466","Type":"ContainerStarted","Data":"4ee23405a3672d03676de2b0280566f8b29b7a44d4b31abe2be7b347d71f2a78"}
Jul 23 04:41:09.028791 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:09.028689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:09.051124 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:09.051046    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh" podStartSLOduration=1.051032691 podStartE2EDuration="1.051032691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:09.049293658 +0000 UTC m=+2729.768825304" watchObservedRunningTime="2026-07-23 04:41:09.051032691 +0000 UTC m=+2729.770564327"
Jul 23 04:41:11.387583 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:11.387555    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w64dx_b08fa11c-9b18-4993-8cc8-cb94dbf83c2a/dns/0.log"
Jul 23 04:41:11.414066 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:11.414040    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w64dx_b08fa11c-9b18-4993-8cc8-cb94dbf83c2a/kube-rbac-proxy/0.log"
Jul 23 04:41:11.551325 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:11.551290    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqk29_cb7d2494-e059-4efb-b6a0-87c444f8b948/dns-node-resolver/0.log"
Jul 23 04:41:12.164865 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:12.164835    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p47ct_1bbe5046-a390-4657-968b-f2943a0213b9/node-ca/0.log"
Jul 23 04:41:13.226401 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:13.226367    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-4zmtr_6bb86030-2683-4b17-bd80-d07ed1845dfd/kube-auth-proxy/0.log"
Jul 23 04:41:14.110622 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:14.110595    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t49dh_6d7b224c-f8af-464c-8239-3cb49da00aac/serve-healthcheck-canary/0.log"
Jul 23 04:41:14.633423 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:14.633389    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7xd2k_7eabeab1-d5db-4b36-83cd-c72e39063356/kube-rbac-proxy/0.log"
Jul 23 04:41:14.656067 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:14.656037    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7xd2k_7eabeab1-d5db-4b36-83cd-c72e39063356/exporter/0.log"
Jul 23 04:41:14.678206 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:14.678183    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7xd2k_7eabeab1-d5db-4b36-83cd-c72e39063356/extractor/0.log"
Jul 23 04:41:15.043635 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:15.043605    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-nkwfh"
Jul 23 04:41:17.756530 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:17.756500    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-4ktqs_841f5b8d-7249-4352-bffa-6c27f35e34d6/manager/0.log"
Jul 23 04:41:17.777582 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:17.777555    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-4ktqs_841f5b8d-7249-4352-bffa-6c27f35e34d6/copy-manifests/0.log"
Jul 23 04:41:19.293970 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:19.293901    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6776597644-jcbwn_596557b9-e5e1-47e8-b0e0-220b0b227715/manager/0.log"
Jul 23 04:41:19.404313 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:19.404276    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-4zmtr_6bb86030-2683-4b17-bd80-d07ed1845dfd/kube-auth-proxy/0.log"
Jul 23 04:41:20.866294 ip-10-0-131-0 kubenswrapper[2563]: I0723 04:41:20.866240    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-4zmtr_6bb86030-2683-4b17-bd80-d07ed1845dfd/kube-auth-proxy/0.log"