Jul 08 02:37:59.333337 ip-10-0-129-88 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 08 02:37:59.333346 ip-10-0-129-88 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 08 02:37:59.333354 ip-10-0-129-88 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 08 02:37:59.333576 ip-10-0-129-88 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 08 02:38:09.582874 ip-10-0-129-88 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 08 02:38:09.582896 ip-10-0-129-88 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2db1a9f7dba945dbb3b8d0a796ee6d28 --
Jul 08 02:40:11.959136 ip-10-0-129-88 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 02:40:12.354796 ip-10-0-129-88 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 02:40:12.354796 ip-10-0-129-88 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 02:40:12.354796 ip-10-0-129-88 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 02:40:12.354796 ip-10-0-129-88 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 02:40:12.354796 ip-10-0-129-88 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 02:40:12.356428 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.356340    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 02:40:12.362892 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362867    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 02:40:12.362892 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362886    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 02:40:12.362892 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362890    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 02:40:12.362892 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362895    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 02:40:12.362892 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362898    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 02:40:12.362892 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362901    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362904    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362907    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362909    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362912    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362915    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362918    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362920    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362923    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362925    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362927    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362930    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362932    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362935    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362937    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362940    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362942    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362945    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362948    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362951    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 02:40:12.363113 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362957    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362960    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362962    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362965    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362968    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362970    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362973    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362975    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362978    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362981    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362983    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362988    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362992    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362996    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.362999    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363002    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363004    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363007    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363010    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 02:40:12.363576 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363012    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363015    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363018    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363020    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363024    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363026    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363029    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363032    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363034    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363037    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363039    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363042    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363044    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363047    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363049    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363052    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363054    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363057    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363059    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363061    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 02:40:12.364054 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363064    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363066    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363069    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363073    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363077    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363080    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363083    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363086    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363089    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363091    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363094    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363097    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363099    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363102    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363104    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363107    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363109    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363112    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363114    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 02:40:12.364526 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363123    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363126    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363129    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363131    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363134    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363136    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363139    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363142    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363144    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363147    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363149    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363152    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363154    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363158    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363163    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363166    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363932    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363940    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363944    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363947    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 02:40:12.364994 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363951    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363954    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363957    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363960    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363963    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363965    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363968    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363971    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363973    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363976    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363978    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363981    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363983    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363988    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363993    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363996    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.363999    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364001    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364004    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 02:40:12.365462 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364007    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364009    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364013    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364016    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364020    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364023    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364026    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364029    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364031    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364034    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364036    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364039    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364043    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364046    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364048    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364051    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364054    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364057    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364059    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 02:40:12.365954 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364063    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364067    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364070    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364073    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364076    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364079    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364083    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364085    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364089    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364091    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364094    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364096    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364099    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364102    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364104    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364108    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364110    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364113    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364115    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364118    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 02:40:12.366416 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364120    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364122    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364125    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364127    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364130    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364132    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364136    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364139    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364141    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364144    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364146    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364149    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364151    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364154    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364156    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364159    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364161    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364164    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364167    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364169    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 02:40:12.367028 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364171    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364174    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364177    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364179    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364182    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364185    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364187    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364190    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364193    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364195    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364198    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364200    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364203    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364205    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364208    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364210    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364213    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364584    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364589    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 02:40:12.367508 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364592    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364595    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364598    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364601    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364603    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364606    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364608    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364611    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364613    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364616    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364619    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364622    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364625    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364627    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364630    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364633    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364636    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364638    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364640    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364643    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 02:40:12.368015 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364646    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364648    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364651    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364653    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364656    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364659    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364661    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364664    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364667    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364669    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364672    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364674    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364677    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364680    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364682    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364685    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364687    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364689    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364692    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364695    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 02:40:12.368504 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364697    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364699    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364702    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364704    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364707    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364711    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364714    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364721    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364724    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364727    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364729    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364732    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364736    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364739    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364742    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364744    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364748    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364751    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 02:40:12.369037 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364754    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364757    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364760    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364762    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364765    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364768    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364770    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364773    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364775    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364778    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364780    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364782    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364785    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364787    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364790    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364792    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364794    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364797    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364799    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364802    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 02:40:12.369469 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364804    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364806    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364809    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364812    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364814    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364817    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364820    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364822    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364825    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364829    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364831    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364834    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364851    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364854    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364856    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364859    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364862    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364867    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.364869    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364938    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364946    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 02:40:12.369967 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364953    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364959    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364964    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364969    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364975    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364981    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364986    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364989    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364992    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364995    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.364999    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365002    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365005    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365008    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365012    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365015    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365019    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365022    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365025    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365028    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365031    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365035    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365038    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 02:40:12.370499 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365041    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365044    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365047    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365050    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365053    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365057    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365060    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365064    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365067    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365070    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365072    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365075    2558 flags.go:64] FLAG: --enable-server="true"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365078    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365083    2558 flags.go:64] FLAG: --event-burst="100"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365086    2558 flags.go:64] FLAG: --event-qps="50"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365088    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365092    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365094    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365098    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365101    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365104    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365107    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365109    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365112    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365116    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 02:40:12.371092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365118    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365121    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365124    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365126    2558 flags.go:64] FLAG: --feature-gates=""
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365130    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365133    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365135    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365138    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365141    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365144    2558 flags.go:64] FLAG: --help="false"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365146    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365149    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365152    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365155    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365158    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365161    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365165    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365167    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365170    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365173    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365176    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365179    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365181    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365184    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 02:40:12.371722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365186    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365189    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365192    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365195    2558 flags.go:64] FLAG: --lock-file=""
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365197    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365200    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365203    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365208    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365213    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365216    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365218    2558 flags.go:64] FLAG: --logging-format="text"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365221    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365224    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365227    2558 flags.go:64] FLAG: --manifest-url=""
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365229    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365234    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365237    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365241    2558 flags.go:64] FLAG: --max-pods="110"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365243    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365246    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365249    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365252    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365254    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365257    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365260    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 02:40:12.372341 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365267    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365270    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365273    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365276    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365279    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365284    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365287    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365290    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365292    2558 flags.go:64] FLAG: --port="10250"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365295    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365298    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0080ed27413a01f45"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365301    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365304    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365307    2558 flags.go:64] FLAG: --register-node="true"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365310    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365313    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365317    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365320    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365323    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365326    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365329    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365332    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365335    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365338    2558 flags.go:64] FLAG: --runonce="false"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365341    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 02:40:12.372990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365344    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365347    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365350    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365353    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365356    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365359    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365362    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365365    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365368    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365371    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365373    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365376    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365379    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365382    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365387    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365390    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365392    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365396    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365399    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365402    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365405    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365407    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365410    2558 flags.go:64] FLAG: --v="2"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365415    2558 flags.go:64] FLAG: --version="false"
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365420    2558 flags.go:64] FLAG: --vmodule=""
Jul 08 02:40:12.373599 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365423    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.365426    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365539    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365543    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365547    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365550    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365554    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365556    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365561    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365566    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365570    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365573    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365576    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365579    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365582    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365585    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365588    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365590    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365593    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 02:40:12.374256 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365596    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365598    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365601    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365603    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365606    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365608    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365611    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365613    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365616    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365619    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365621    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365625    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365627    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365630    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365633    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365635    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365638    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365641    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365643    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365646    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 02:40:12.374725 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365648    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365651    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365656    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365658    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365661    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365663    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365665    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365668    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365671    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365673    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365677    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365680    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365683    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365686    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365688    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365691    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365693    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365696    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365698    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 02:40:12.375225 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365700    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365703    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365705    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365708    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365713    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365716    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365719    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365729    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365732    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365734    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365737    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365739    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365742    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365745    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365747    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365751    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365753    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365756    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365758    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365761    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 02:40:12.375685 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365763    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365766    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365768    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365771    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365773    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365776    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365778    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365781    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365783    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365786    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365788    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365790    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365793    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365795    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365798    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365800    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365804    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365806    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365809    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365811    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 02:40:12.376183 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365814    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365817    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.365820    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.366505    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.373179    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.373196    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.373215    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.373220    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373276    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373281    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373284    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373287    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373290    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373292    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 02:40:12.376727 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373295    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373298    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373301    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373304    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373306    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373309    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373312    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373314    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373317    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373319    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373322    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373325    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373327    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373329    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373332    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373334    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373336    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373339    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373341    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373343    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 02:40:12.377151 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373346    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373349    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373352    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373355    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373358    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373361    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373363    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373366    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373368    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373371    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373373    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373376    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373378    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373382    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373386    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373389    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373392    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373394    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373397    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373400    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 02:40:12.377642 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373403    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373406    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373408    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373411    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373413    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373416    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373419    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373422    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373425    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373427    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373430    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373434    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373437    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373440    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373443    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373446    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373449    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373452    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373454    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 02:40:12.378447 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373457    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373459    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373461    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373464    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373467    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373471    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373474    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373476    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373479    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373481    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373483    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373486    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373488    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373491    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373494    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373496    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373498    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373501    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373503    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 02:40:12.379116 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373506    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373510    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373514    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373517    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373520    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373523    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373526    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373528    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373532    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373535    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373537    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373540    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373543    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373546    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373548    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 02:40:12.379651 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.373554    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373672    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373677    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373680    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373683    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373686    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373688    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373691    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373693    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373696    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373698    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373701    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373703    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373705    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373708    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373710    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373713    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373716    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373718    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 02:40:12.380104 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373720    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373723    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373725    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373728    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373730    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373732    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373736    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373741    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373743    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373746    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373750    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373752    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373755    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373758    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373760    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373763    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373766    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373768    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373771    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 02:40:12.380557 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373773    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373775    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373778    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373780    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373783    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373785    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373787    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373790    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373792    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373795    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373797    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373800    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373803    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373806    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373808    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373811    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373813    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373815    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373818    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373820    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 02:40:12.381191 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373823    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373825    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373828    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373830    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373833    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373850    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373853    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373855    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373858    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373861    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373863    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373866    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373868    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373871    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373873    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373876    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373878    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373881    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373883    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373885    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 02:40:12.381940 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373888    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373891    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373893    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373896    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373899    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373901    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373904    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373906    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373909    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373911    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373914    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373917    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373921    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373924    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373927    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373930    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373933    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373937    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373940    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 02:40:12.382718 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373942    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373945    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:12.373947    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.373952    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.374070    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.376286    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.377595    2558 server.go:1019] "Starting client certificate rotation"
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.377718    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.377762    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 02:40:12.383361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.379902    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 02:40:12.383729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.383707    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 02:40:12.393544 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.393527    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 08 02:40:12.397214 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.397200    2558 log.go:25] "Validated CRI v1 image API"
Jul 08 02:40:12.398369 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.398343    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 02:40:12.402185 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.402165    2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8eb8e74f-638a-4107-b467-42389c57052c:/dev/nvme0n1p4 ddc5ead9-a60c-438a-9c04-865e16f0aaa8:/dev/nvme0n1p3]
Jul 08 02:40:12.402259 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.402184    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 02:40:12.406973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.406856    2558 manager.go:216] Machine: {Timestamp:2026-07-08 02:40:12.405745041 +0000 UTC m=+0.333606643 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101563 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26f5033d142c87125bc063f1c18be1 SystemUUID:ec26f503-3d14-2c87-125b-c063f1c18be1 BootID:2db1a9f7-dba9-45db-b3b8-d0a796ee6d28 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:15:c9:02:1c:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:15:c9:02:1c:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:2d:c2:b2:6d:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 02:40:12.406973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.406966    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 02:40:12.407081 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.407042    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 02:40:12.408776 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.408759    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 02:40:12.410555 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.410531    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 02:40:12.410707 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.410560    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-88.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 02:40:12.411849 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.411825    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 02:40:12.411884 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.411855    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 02:40:12.411884 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.411863    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 02:40:12.412539 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.412528    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 02:40:12.413736 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.413726    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 02:40:12.417105 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417092    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 02:40:12.417237 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417226    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 02:40:12.417275 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417261    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 02:40:12.417275 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417272    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 02:40:12.417349 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417284    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 02:40:12.417349 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417292    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 08 02:40:12.417349 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417301    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 02:40:12.417780 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417758    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 02:40:12.417824 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.417810    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 02:40:12.420331 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.420310    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 02:40:12.422013 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.422000    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 02:40:12.422085 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.422030    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 02:40:12.423195 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423185    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 02:40:12.423229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423200    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 02:40:12.423229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423207    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 02:40:12.423229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423212    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 02:40:12.423229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423221    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 02:40:12.423229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423226    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 02:40:12.423359 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423233    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 02:40:12.423359 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423238    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 02:40:12.423359 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423244    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 02:40:12.423359 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423250    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 02:40:12.423359 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423259    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 02:40:12.423359 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.423265    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 02:40:12.425135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.425123    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 02:40:12.425135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.425135    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 02:40:12.428385 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.428368    2558 server.go:1268] "Started kubelet"
Jul 08 02:40:12.428479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.428448    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 02:40:12.429282 ip-10-0-129-88 systemd[1]: Started Kubernetes Kubelet.
Jul 08 02:40:12.431498 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.431418    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 02:40:12.431593 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.431532    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 02:40:12.432373 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.432346    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-88.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 02:40:12.432714 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.432690    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 02:40:12.432870 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.432831    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-88.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 02:40:12.433239 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.433211    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 02:40:12.433880 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.433863    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 02:40:12.436979 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.436952    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 02:40:12.439233 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.438327    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-88.ec2.internal.18c0302ab55e6e37  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-88.ec2.internal,UID:ip-10-0-129-88.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-88.ec2.internal,},FirstTimestamp:2026-07-08 02:40:12.428340791 +0000 UTC m=+0.356202391,LastTimestamp:2026-07-08 02:40:12.428340791 +0000 UTC m=+0.356202391,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-88.ec2.internal,}"
Jul 08 02:40:12.440155 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.440120    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mqlm2"
Jul 08 02:40:12.440155 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.440132    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 02:40:12.440155 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.440132    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 02:40:12.440710 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.440696    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:12.441160 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441144    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 02:40:12.441282 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441263    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 02:40:12.441377 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441144    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 02:40:12.441430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441400    2558 factory.go:55] Registering systemd factory
Jul 08 02:40:12.441430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441415    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 08 02:40:12.441430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441418    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 02:40:12.441430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441425    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 02:40:12.441698 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441680    2558 factory.go:153] Registering CRI-O factory
Jul 08 02:40:12.441757 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441703    2558 factory.go:223] Registration of the crio container factory successfully
Jul 08 02:40:12.441802 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441761    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 02:40:12.441802 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441787    2558 factory.go:103] Registering Raw factory
Jul 08 02:40:12.441907 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.441803    2558 manager.go:1196] Started watching for new ooms in manager
Jul 08 02:40:12.442483 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.442467    2558 manager.go:349] Starting recovery of all containers
Jul 08 02:40:12.445243 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.445212    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 02:40:12.445370 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.445349    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-88.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 02:40:12.448531 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.448510    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mqlm2"
Jul 08 02:40:12.453098 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.452972    2558 manager.go:354] Recovery completed
Jul 08 02:40:12.457101 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.457088    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.459044 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459028    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.459123 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459056    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.459123 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459066    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.459539 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459524    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 02:40:12.459539 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459536    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 02:40:12.459669 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459537    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.459669 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.459554    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 02:40:12.460313 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.460295    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.460371 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.460331    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.460371 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.460346    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.460813 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.460801    2558 policy_none.go:49] "None policy: Start"
Jul 08 02:40:12.460866 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.460816    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 02:40:12.460866 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.460827    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 02:40:12.461329 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.461251    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-88.ec2.internal.18c0302ab732eb2c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-88.ec2.internal,UID:ip-10-0-129-88.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-88.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-88.ec2.internal,},FirstTimestamp:2026-07-08 02:40:12.459043628 +0000 UTC m=+0.386905230,LastTimestamp:2026-07-08 02:40:12.459043628 +0000 UTC m=+0.386905230,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-88.ec2.internal,}"
Jul 08 02:40:12.461879 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.461867    2558 policy_none.go:47] "Start"
Jul 08 02:40:12.503490 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503466    2558 manager.go:337] "Starting Device Plugin manager"
Jul 08 02:40:12.503623 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.503504    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 02:40:12.503623 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503518    2558 server.go:87] "Starting device plugin registration server"
Jul 08 02:40:12.503809 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503796    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 02:40:12.503864 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503813    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 02:40:12.503918 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503888    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 02:40:12.503975 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503963    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 02:40:12.504022 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.503976    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 02:40:12.504607 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.504584    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 02:40:12.504670 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.504647    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:12.604944 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.604827    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.606209 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.605873    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.606209 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.605899    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.606209 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.605909    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.606209 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.605935    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.611933 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.611900    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 02:40:12.613316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.613301    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 02:40:12.613380 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.613338    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 02:40:12.613380 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.613358    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 02:40:12.613476 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.613398    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 02:40:12.614736 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.614715    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.614833 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.614746    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-88.ec2.internal\": node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:12.615826 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.615813    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 02:40:12.688004 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.687968    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:12.714100 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.714058    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"]
Jul 08 02:40:12.714200 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.714148    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.715714 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.715698    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.715807 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.715728    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.715807 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.715741    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.716995 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.716982    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.717107 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717093    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.717170 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717126    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.717851 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717816    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.717932 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717859    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.717932 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717870    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.717932 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717927    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.718023 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717951    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.718023 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.717962    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.718887 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.718872    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.718928 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.718908    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 02:40:12.719644 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.719626    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 02:40:12.719746 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.719653    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 02:40:12.719746 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.719670    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeHasSufficientPID"
Jul 08 02:40:12.742956 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.742928    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb75151fa942c83c5acbdad9dea7c0bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal\" (UID: \"bb75151fa942c83c5acbdad9dea7c0bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.742956 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.742955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb75151fa942c83c5acbdad9dea7c0bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal\" (UID: \"bb75151fa942c83c5acbdad9dea7c0bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.743095 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.742973    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1fa326e2985d0d87989ce38a6f8caaeb-config\") pod \"kube-apiserver-proxy-ip-10-0-129-88.ec2.internal\" (UID: \"1fa326e2985d0d87989ce38a6f8caaeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.745457 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.745443    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-88.ec2.internal\" not found" node="ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.749747 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.749729    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-88.ec2.internal\" not found" node="ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.788646 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.788604    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:12.843854 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.843814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb75151fa942c83c5acbdad9dea7c0bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal\" (UID: \"bb75151fa942c83c5acbdad9dea7c0bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.844016 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.843868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb75151fa942c83c5acbdad9dea7c0bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal\" (UID: \"bb75151fa942c83c5acbdad9dea7c0bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.844016 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.843887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1fa326e2985d0d87989ce38a6f8caaeb-config\") pod \"kube-apiserver-proxy-ip-10-0-129-88.ec2.internal\" (UID: \"1fa326e2985d0d87989ce38a6f8caaeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.844016 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.843916    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb75151fa942c83c5acbdad9dea7c0bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal\" (UID: \"bb75151fa942c83c5acbdad9dea7c0bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.844016 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.843952    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb75151fa942c83c5acbdad9dea7c0bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal\" (UID: \"bb75151fa942c83c5acbdad9dea7c0bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.844016 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:12.843926    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1fa326e2985d0d87989ce38a6f8caaeb-config\") pod \"kube-apiserver-proxy-ip-10-0-129-88.ec2.internal\" (UID: \"1fa326e2985d0d87989ce38a6f8caaeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:12.889144 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.889064    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:12.990746 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:12.990713    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.047994 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.047938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:13.053117 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.053089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:13.091616 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.091551    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.195194 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.195152    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.296119 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.296068    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.378451 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.378412    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 02:40:13.379097 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.378583    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 02:40:13.397111 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.397073    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.450754 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.450649    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 02:35:12 +0000 UTC" deadline="2027-12-09 15:25:53.121387333 +0000 UTC"
Jul 08 02:40:13.450754 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.450678    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12468h45m39.670712665s"
Jul 08 02:40:13.498101 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.498060    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.599023 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.598982    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.700176 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.700142    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.717375 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:13.717332    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa326e2985d0d87989ce38a6f8caaeb.slice/crio-b5ca2b954d60e2287de7123616ca3519c1b37740105ece3ce8d246d71d2d5192 WatchSource:0}: Error finding container b5ca2b954d60e2287de7123616ca3519c1b37740105ece3ce8d246d71d2d5192: Status 404 returned error can't find the container with id b5ca2b954d60e2287de7123616ca3519c1b37740105ece3ce8d246d71d2d5192
Jul 08 02:40:13.717656 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:13.717642    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb75151fa942c83c5acbdad9dea7c0bc.slice/crio-b36ba034ef0bc206be15dfc85e5fb3a6e2242befc4037b047ac2efd2f072c7a5 WatchSource:0}: Error finding container b36ba034ef0bc206be15dfc85e5fb3a6e2242befc4037b047ac2efd2f072c7a5: Status 404 returned error can't find the container with id b36ba034ef0bc206be15dfc85e5fb3a6e2242befc4037b047ac2efd2f072c7a5
Jul 08 02:40:13.723481 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.723464    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 02:40:13.734178 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.734157    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 02:40:13.801086 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.801038    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.901731 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:13.901698    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-88.ec2.internal\" not found"
Jul 08 02:40:13.953292 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:13.953264    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 02:40:14.015755 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.015670    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 02:40:14.041851 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.041803    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:14.055113 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.055087    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 02:40:14.056294 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.056278    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal"
Jul 08 02:40:14.066051 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.066017    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 02:40:14.418970 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.418659    2558 apiserver.go:52] "Watching apiserver"
Jul 08 02:40:14.425850 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.425819    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 02:40:14.426503 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.426391    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-tzzq2","kube-system/global-pull-secret-syncer-rvgzh","kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gxfd5","openshift-dns/node-resolver-p259t","openshift-image-registry/node-ca-tts6l","openshift-multus/multus-b5rqr","openshift-network-diagnostics/network-check-target-hkrtz","openshift-network-operator/iptables-alerter-5fvmr","kube-system/konnectivity-agent-wg2vj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal","openshift-multus/multus-additional-cni-plugins-7sd9k","openshift-multus/network-metrics-daemon-xkg96"]
Jul 08 02:40:14.428173 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.428143    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.429932 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.429907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.431821 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.431683    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:14.431821 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.431766    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:14.432436 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432105    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.432436 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432287    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 02:40:14.432436 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432351    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 02:40:14.432674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432641    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zf94f\""
Jul 08 02:40:14.432674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432663    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432868    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.432978    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433062    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433144    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433263    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pmc7j\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433308    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433375    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 02:40:14.433573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433568    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.434049 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.433680    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.435224 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.434940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.436496 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.436475    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.436727 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.436710    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.438156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.437925    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 02:40:14.438156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.437938    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.438156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.437980    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sgdzn\""
Jul 08 02:40:14.438365 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.438182    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vvbkw\""
Jul 08 02:40:14.438365 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.438208    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.439376    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.439453    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tvnb4\""
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.439576    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.439782    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.440857    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.441027    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.441100    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.441297 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.441185    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8658c\""
Jul 08 02:40:14.448377 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.448357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.451383 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.449594    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.451383 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.450735    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.452248 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.452225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.452375 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.452331    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:14.452521 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.452487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-ovn\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.452590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.452551    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-cni-netd\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.452590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.452585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysctl-conf\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.452700 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.452619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb9px\" (UniqueName: \"kubernetes.io/projected/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-kube-api-access-pb9px\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.452802 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.452695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-k8s-cni-cncf-io\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.453479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453456    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/488bd2fd-661e-4813-9b10-29d42a01e530-host-slash\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.453581 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453496    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-log-socket\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.453581 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-cni-bin\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.453581 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453561    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-netns\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.453751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-hostroot\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.453751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-run-netns\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.453751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-host\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.453751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-cni-multus\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.453751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453687    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-hosts-file\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.453751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxl9g\" (UniqueName: \"kubernetes.io/projected/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-kube-api-access-wxl9g\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-cnibin\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453785    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-tmp\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453809    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhmv9\" (UniqueName: \"kubernetes.io/projected/341b33aa-f143-402d-99e0-39965d30bd04-kube-api-access-vhmv9\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453847    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-systemd-units\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453895    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c4sj\" (UniqueName: \"kubernetes.io/projected/439740dd-c52f-4805-9f49-7f1b0086f066-kube-api-access-5c4sj\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-tmp-dir\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453951    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-slash\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.453980    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454004    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-systemd\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454028    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-tuned\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454053    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.454077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454078    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f6f7f3c-3e93-4a58-94f0-16e462e437e5-konnectivity-ca\") pod \"konnectivity-agent-wg2vj\" (UID: \"6f6f7f3c-3e93-4a58-94f0-16e462e437e5\") " pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/488bd2fd-661e-4813-9b10-29d42a01e530-iptables-alerter-script\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-var-lib-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454161    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-system-cni-dir\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454216    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-cnibin\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-multus-certs\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454293    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovnkube-script-lib\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454320    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454344    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysconfig\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-sys\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-var-lib-kubelet\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454454    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4sp4\" (UniqueName: \"kubernetes.io/projected/30e3fc41-317a-4aa5-aa4c-c76011d1d220-kube-api-access-n4sp4\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454482    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-env-overrides\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovn-node-metrics-cert\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.454729 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454557    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-cni-binary-copy\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454583    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-cni-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454630    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-socket-dir-parent\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454670    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454704    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-cni-bin\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-kubelet\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454761    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtvmz\" (UniqueName: \"kubernetes.io/projected/488bd2fd-661e-4813-9b10-29d42a01e530-kube-api-access-xtvmz\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454794    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-etc-kubernetes\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-etc-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-node-log\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454954    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-os-release\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.454977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-modprobe-d\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455000    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-system-cni-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/341b33aa-f143-402d-99e0-39965d30bd04-multus-daemon-config\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-kubelet\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-run\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-lib-modules\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.455561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-os-release\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/341b33aa-f143-402d-99e0-39965d30bd04-cni-binary-copy\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f6f7f3c-3e93-4a58-94f0-16e462e437e5-agent-certs\") pod \"konnectivity-agent-wg2vj\" (UID: \"6f6f7f3c-3e93-4a58-94f0-16e462e437e5\") " pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455274    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovnkube-config\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-conf-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30e3fc41-317a-4aa5-aa4c-c76011d1d220-host\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455359    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-systemd\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455384    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snlv2\" (UniqueName: \"kubernetes.io/projected/e4c49f8e-fe3b-4cab-af44-166d9693057c-kube-api-access-snlv2\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455409    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-kubernetes\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455457    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysctl-d\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455480    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30e3fc41-317a-4aa5-aa4c-c76011d1d220-serviceca\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455654    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.455981    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x9l6g\""
Jul 08 02:40:14.456415 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.456053    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 02:40:14.457404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.456751    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 02:40:14.457404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.456902    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4qtjm\""
Jul 08 02:40:14.457404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.457002    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 02:40:14.457404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.457122    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.457404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.457331    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 02:40:14.457711 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.457573    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7pwjq\""
Jul 08 02:40:14.457769 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.457755    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 02:40:14.458456 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.458011    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 02:40:14.459560 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.459539    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:14.459661 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.459638    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:14.467364 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.467319    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 02:40:14.486804 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.486778    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6bvp7"
Jul 08 02:40:14.496220 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.496200    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6bvp7"
Jul 08 02:40:14.542374 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.542344    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 02:40:14.557361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-env-overrides\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.557514 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557367    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovn-node-metrics-cert\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.557514 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-cni-binary-copy\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.557514 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-cni-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557514 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-socket-dir-parent\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557514 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557492    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-sys-fs\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557548    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-cni-bin\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557572    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-kubelet\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtvmz\" (UniqueName: \"kubernetes.io/projected/488bd2fd-661e-4813-9b10-29d42a01e530-kube-api-access-xtvmz\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557628    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-etc-kubernetes\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557660    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-etc-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-node-log\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-os-release\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-modprobe-d\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557759    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-system-cni-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/341b33aa-f143-402d-99e0-39965d30bd04-multus-daemon-config\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.557788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-kubelet\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-run\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-lib-modules\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-os-release\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557857    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/341b33aa-f143-402d-99e0-39965d30bd04-cni-binary-copy\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557873    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f6f7f3c-3e93-4a58-94f0-16e462e437e5-agent-certs\") pod \"konnectivity-agent-wg2vj\" (UID: \"6f6f7f3c-3e93-4a58-94f0-16e462e437e5\") " pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-socket-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovnkube-config\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-conf-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557928    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30e3fc41-317a-4aa5-aa4c-c76011d1d220-host\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-systemd\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.557999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snlv2\" (UniqueName: \"kubernetes.io/projected/e4c49f8e-fe3b-4cab-af44-166d9693057c-kube-api-access-snlv2\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558016    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558032    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-kubernetes\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558037    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-env-overrides\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysctl-d\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30e3fc41-317a-4aa5-aa4c-c76011d1d220-serviceca\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.558399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558173    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysctl-d\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-registration-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558210    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-kubelet\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558219    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-ovn\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558241    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-run\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-cni-netd\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysctl-conf\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558313    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-lib-modules\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb9px\" (UniqueName: \"kubernetes.io/projected/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-kube-api-access-pb9px\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-k8s-cni-cncf-io\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/488bd2fd-661e-4813-9b10-29d42a01e530-host-slash\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558421    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzzjg\" (UniqueName: \"kubernetes.io/projected/bff0db38-2156-4511-afe0-0786c5200158-kube-api-access-bzzjg\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-log-socket\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-cni-bin\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-netns\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558506    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-cni-binary-copy\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.559316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-hostroot\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558573    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/10e31942-6737-4b44-a21c-3009611e2f37-dbus\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558580    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-etc-kubernetes\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-run-netns\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-host\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558667    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-etc-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-cni-multus\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-hosts-file\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558731    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-node-log\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558754    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxl9g\" (UniqueName: \"kubernetes.io/projected/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-kube-api-access-wxl9g\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558777    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-os-release\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558781    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-cnibin\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558812    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-tmp\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558829    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-system-cni-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhmv9\" (UniqueName: \"kubernetes.io/projected/341b33aa-f143-402d-99e0-39965d30bd04-kube-api-access-vhmv9\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558867    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-conf-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9jvc\" (UniqueName: \"kubernetes.io/projected/0cb017c8-2eee-465b-9639-575474c42599-kube-api-access-x9jvc\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558891    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-socket-dir-parent\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558937    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-systemd-units\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4sj\" (UniqueName: \"kubernetes.io/projected/439740dd-c52f-4805-9f49-7f1b0086f066-kube-api-access-5c4sj\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558991    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30e3fc41-317a-4aa5-aa4c-c76011d1d220-host\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-tmp-dir\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559026    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-modprobe-d\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559042    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559047    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-systemd\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559080    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-cni-bin\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559139    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-run-netns\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559204    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-host\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559252    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-cni-multus\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559309    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-hosts-file\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559456    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/341b33aa-f143-402d-99e0-39965d30bd04-cni-binary-copy\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559533    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30e3fc41-317a-4aa5-aa4c-c76011d1d220-serviceca\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558775    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-multus-cni-dir\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559579    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-var-lib-kubelet\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559638    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-ovn\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.560674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559671    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-cni-netd\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559756    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysctl-conf\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559777    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559870    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-slash\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559902    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559950    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-systemd\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.559981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-tuned\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.560042    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-k8s-cni-cncf-io\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.560082    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.560138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f6f7f3c-3e93-4a58-94f0-16e462e437e5-konnectivity-ca\") pod \"konnectivity-agent-wg2vj\" (UID: \"6f6f7f3c-3e93-4a58-94f0-16e462e437e5\") " pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.561235    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/488bd2fd-661e-4813-9b10-29d42a01e530-iptables-alerter-script\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.561430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.561284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-var-lib-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565414    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-kubernetes\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565518    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/341b33aa-f143-402d-99e0-39965d30bd04-multus-daemon-config\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565587    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-cnibin\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565650    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-slash\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565739    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-run-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565849    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-systemd\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565862    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565934    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-systemd-units\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-system-cni-dir\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565986    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-system-cni-dir\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.565985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-cnibin\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566036    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-cnibin\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-multus-certs\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566085    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-multus-certs\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-device-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.558938    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-os-release\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-etc-selinux\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.566737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566436    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-tmp-dir\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566507    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/439740dd-c52f-4805-9f49-7f1b0086f066-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566662    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/488bd2fd-661e-4813-9b10-29d42a01e530-host-slash\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566743    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-log-socket\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566812    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-cni-bin\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566863    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-host-run-netns\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.566933    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/341b33aa-f143-402d-99e0-39965d30bd04-hostroot\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567044    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f6f7f3c-3e93-4a58-94f0-16e462e437e5-konnectivity-ca\") pod \"konnectivity-agent-wg2vj\" (UID: \"6f6f7f3c-3e93-4a58-94f0-16e462e437e5\") " pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567065    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f6f7f3c-3e93-4a58-94f0-16e462e437e5-agent-certs\") pod \"konnectivity-agent-wg2vj\" (UID: \"6f6f7f3c-3e93-4a58-94f0-16e462e437e5\") " pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567101    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/10e31942-6737-4b44-a21c-3009611e2f37-kubelet-config\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567166    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.567706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567219    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovnkube-script-lib\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567723    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/488bd2fd-661e-4813-9b10-29d42a01e530-iptables-alerter-script\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567773    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysconfig\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-sys\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567903    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-var-lib-kubelet\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.567943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4sp4\" (UniqueName: \"kubernetes.io/projected/30e3fc41-317a-4aa5-aa4c-c76011d1d220-kube-api-access-n4sp4\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.568022    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-sysconfig\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.568117    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/439740dd-c52f-4805-9f49-7f1b0086f066-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.568293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.568292    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-sys\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.568740 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.568462    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovnkube-script-lib\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.568740 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.568547    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-var-lib-openvswitch\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.568740 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.568605    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-var-lib-kubelet\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.569258 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.569023    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c49f8e-fe3b-4cab-af44-166d9693057c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.569258 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.569145    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovnkube-config\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.570131 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.570108    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4c49f8e-fe3b-4cab-af44-166d9693057c-ovn-node-metrics-cert\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.570575 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.570492    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-tmp\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.570819 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.570748    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb9px\" (UniqueName: \"kubernetes.io/projected/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-kube-api-access-pb9px\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.572338 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.571969    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:14.572338 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.571997    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:14.572338 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.572018    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:14.572338 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.572112    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:15.072082222 +0000 UTC m=+2.999943827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:14.572788 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.572671    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6bd750e4-0d2e-4de2-99f8-41b0d0c0d663-etc-tuned\") pod \"tuned-gxfd5\" (UID: \"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663\") " pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.573738 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.573717    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtvmz\" (UniqueName: \"kubernetes.io/projected/488bd2fd-661e-4813-9b10-29d42a01e530-kube-api-access-xtvmz\") pod \"iptables-alerter-5fvmr\" (UID: \"488bd2fd-661e-4813-9b10-29d42a01e530\") " pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.575530 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.575493    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhmv9\" (UniqueName: \"kubernetes.io/projected/341b33aa-f143-402d-99e0-39965d30bd04-kube-api-access-vhmv9\") pod \"multus-b5rqr\" (UID: \"341b33aa-f143-402d-99e0-39965d30bd04\") " pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.577151 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.577124    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxl9g\" (UniqueName: \"kubernetes.io/projected/b4647bc1-61bf-4ef6-9547-559ff6f3ec49-kube-api-access-wxl9g\") pod \"node-resolver-p259t\" (UID: \"b4647bc1-61bf-4ef6-9547-559ff6f3ec49\") " pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.577413 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.577393    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snlv2\" (UniqueName: \"kubernetes.io/projected/e4c49f8e-fe3b-4cab-af44-166d9693057c-kube-api-access-snlv2\") pod \"ovnkube-node-tzzq2\" (UID: \"e4c49f8e-fe3b-4cab-af44-166d9693057c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.583468 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.583443    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c4sj\" (UniqueName: \"kubernetes.io/projected/439740dd-c52f-4805-9f49-7f1b0086f066-kube-api-access-5c4sj\") pod \"multus-additional-cni-plugins-7sd9k\" (UID: \"439740dd-c52f-4805-9f49-7f1b0086f066\") " pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.583569 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.583543    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4sp4\" (UniqueName: \"kubernetes.io/projected/30e3fc41-317a-4aa5-aa4c-c76011d1d220-kube-api-access-n4sp4\") pod \"node-ca-tts6l\" (UID: \"30e3fc41-317a-4aa5-aa4c-c76011d1d220\") " pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.620919 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.620817    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal" event={"ID":"bb75151fa942c83c5acbdad9dea7c0bc","Type":"ContainerStarted","Data":"b36ba034ef0bc206be15dfc85e5fb3a6e2242befc4037b047ac2efd2f072c7a5"}
Jul 08 02:40:14.622874 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.622840    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal" event={"ID":"1fa326e2985d0d87989ce38a6f8caaeb","Type":"ContainerStarted","Data":"b5ca2b954d60e2287de7123616ca3519c1b37740105ece3ce8d246d71d2d5192"}
Jul 08 02:40:14.653367 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.653334    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 02:40:14.668722 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-socket-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.668924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.668924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-registration-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.668924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzzjg\" (UniqueName: \"kubernetes.io/projected/bff0db38-2156-4511-afe0-0786c5200158-kube-api-access-bzzjg\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.668924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/10e31942-6737-4b44-a21c-3009611e2f37-dbus\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.668924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9jvc\" (UniqueName: \"kubernetes.io/projected/0cb017c8-2eee-465b-9639-575474c42599-kube-api-access-x9jvc\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:14.668924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668916    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.669316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:14.669316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.668980    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-device-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669005    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-etc-selinux\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/10e31942-6737-4b44-a21c-3009611e2f37-kubelet-config\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.669316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-sys-fs\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669316 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669211    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-sys-fs\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669662 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669341    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-socket-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669662 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669399    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669662 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669448    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-registration-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.669874 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.669841    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/10e31942-6737-4b44-a21c-3009611e2f37-dbus\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.670199 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.670166    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:14.670295 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.670243    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:15.170217156 +0000 UTC m=+3.098078748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:14.670484 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.670449    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:14.670594 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:14.670499    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:15.170484968 +0000 UTC m=+3.098346563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:14.670594 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.670557    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-device-dir\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.670720 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.670662    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bff0db38-2156-4511-afe0-0786c5200158-etc-selinux\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.670775 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.670732    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/10e31942-6737-4b44-a21c-3009611e2f37-kubelet-config\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:14.683256 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.683221    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzzjg\" (UniqueName: \"kubernetes.io/projected/bff0db38-2156-4511-afe0-0786c5200158-kube-api-access-bzzjg\") pod \"aws-ebs-csi-driver-node-ldrhl\" (UID: \"bff0db38-2156-4511-afe0-0786c5200158\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.687138 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.687054    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9jvc\" (UniqueName: \"kubernetes.io/projected/0cb017c8-2eee-465b-9639-575474c42599-kube-api-access-x9jvc\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:14.750299 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.750263    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:14.761065 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.761029    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c49f8e_fe3b_4cab_af44_166d9693057c.slice/crio-520f868e54d9ce3ac5e8ced43b644c73995b9ae2778b2f3dc9f7d09c5832c29f WatchSource:0}: Error finding container 520f868e54d9ce3ac5e8ced43b644c73995b9ae2778b2f3dc9f7d09c5832c29f: Status 404 returned error can't find the container with id 520f868e54d9ce3ac5e8ced43b644c73995b9ae2778b2f3dc9f7d09c5832c29f
Jul 08 02:40:14.761427 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.761405    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7sd9k"
Jul 08 02:40:14.772935 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.772885    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl"
Jul 08 02:40:14.773514 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.773439    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439740dd_c52f_4805_9f49_7f1b0086f066.slice/crio-18a05402eb87a2103b8bbffb7445117005fc5d804a8881ecac537657f73e72c7 WatchSource:0}: Error finding container 18a05402eb87a2103b8bbffb7445117005fc5d804a8881ecac537657f73e72c7: Status 404 returned error can't find the container with id 18a05402eb87a2103b8bbffb7445117005fc5d804a8881ecac537657f73e72c7
Jul 08 02:40:14.778973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.778950    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gxfd5"
Jul 08 02:40:14.783684 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.783640    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff0db38_2156_4511_afe0_0786c5200158.slice/crio-cf024bac6e2e24398688a08c29f88ecfba4e980834f632c3e84744a224c1c8c5 WatchSource:0}: Error finding container cf024bac6e2e24398688a08c29f88ecfba4e980834f632c3e84744a224c1c8c5: Status 404 returned error can't find the container with id cf024bac6e2e24398688a08c29f88ecfba4e980834f632c3e84744a224c1c8c5
Jul 08 02:40:14.788838 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.788813    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b5rqr"
Jul 08 02:40:14.789172 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.789150    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd750e4_0d2e_4de2_99f8_41b0d0c0d663.slice/crio-d0a11ab9b09e09e8900e620849b2a15c583199969e78835b45f1ba195954025f WatchSource:0}: Error finding container d0a11ab9b09e09e8900e620849b2a15c583199969e78835b45f1ba195954025f: Status 404 returned error can't find the container with id d0a11ab9b09e09e8900e620849b2a15c583199969e78835b45f1ba195954025f
Jul 08 02:40:14.797520 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.797497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tts6l"
Jul 08 02:40:14.799100 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.799066    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341b33aa_f143_402d_99e0_39965d30bd04.slice/crio-43c8de396ecfc7cc19f2e2b85d64826c6b7619a32a94e3fa34fcb50346b16708 WatchSource:0}: Error finding container 43c8de396ecfc7cc19f2e2b85d64826c6b7619a32a94e3fa34fcb50346b16708: Status 404 returned error can't find the container with id 43c8de396ecfc7cc19f2e2b85d64826c6b7619a32a94e3fa34fcb50346b16708
Jul 08 02:40:14.803360 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.803339    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p259t"
Jul 08 02:40:14.807937 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.807870    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e3fc41_317a_4aa5_aa4c_c76011d1d220.slice/crio-5469a0843db2ad9a228651103349ec42aeada9d65485070e194f588c5d33b897 WatchSource:0}: Error finding container 5469a0843db2ad9a228651103349ec42aeada9d65485070e194f588c5d33b897: Status 404 returned error can't find the container with id 5469a0843db2ad9a228651103349ec42aeada9d65485070e194f588c5d33b897
Jul 08 02:40:14.813622 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.813586    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5fvmr"
Jul 08 02:40:14.814953 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.814871    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4647bc1_61bf_4ef6_9547_559ff6f3ec49.slice/crio-14479a461cdffbf810038616944a2efa423b77588da4f412d00e4a3ed59fda21 WatchSource:0}: Error finding container 14479a461cdffbf810038616944a2efa423b77588da4f412d00e4a3ed59fda21: Status 404 returned error can't find the container with id 14479a461cdffbf810038616944a2efa423b77588da4f412d00e4a3ed59fda21
Jul 08 02:40:14.818719 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:14.818695    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:14.827282 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.827248    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488bd2fd_661e_4813_9b10_29d42a01e530.slice/crio-a477349f3d5cd93804492f3a059f5f0dd4615720ca3e7c82b37003bb6230bfb9 WatchSource:0}: Error finding container a477349f3d5cd93804492f3a059f5f0dd4615720ca3e7c82b37003bb6230bfb9: Status 404 returned error can't find the container with id a477349f3d5cd93804492f3a059f5f0dd4615720ca3e7c82b37003bb6230bfb9
Jul 08 02:40:14.830410 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:14.830385    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6f7f3c_3e93_4a58_94f0_16e462e437e5.slice/crio-3d0b24a85a0e89253dd9edb6d40d0565c0e9ef5c37864674891fcfb99b63ea76 WatchSource:0}: Error finding container 3d0b24a85a0e89253dd9edb6d40d0565c0e9ef5c37864674891fcfb99b63ea76: Status 404 returned error can't find the container with id 3d0b24a85a0e89253dd9edb6d40d0565c0e9ef5c37864674891fcfb99b63ea76
Jul 08 02:40:15.172734 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.172690    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:15.172937 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.172745    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:15.172937 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.172804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:15.172937 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.172905    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.172974    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.173016    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:16.172992742 +0000 UTC m=+4.100854348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.173021    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.173042    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.173043    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:16.173024682 +0000 UTC m=+4.100886287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.173054    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:15.173116 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.173096    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:16.173082199 +0000 UTC m=+4.100943802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:15.500259 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.500157    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 02:35:14 +0000 UTC" deadline="2028-02-04 17:19:02.790325297 +0000 UTC"
Jul 08 02:40:15.500259 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.500195    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13838h38m47.290133871s"
Jul 08 02:40:15.614050 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.614008    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:15.614209 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:15.614150    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:15.626988 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.626910    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wg2vj" event={"ID":"6f6f7f3c-3e93-4a58-94f0-16e462e437e5","Type":"ContainerStarted","Data":"3d0b24a85a0e89253dd9edb6d40d0565c0e9ef5c37864674891fcfb99b63ea76"}
Jul 08 02:40:15.628557 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.628520    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tts6l" event={"ID":"30e3fc41-317a-4aa5-aa4c-c76011d1d220","Type":"ContainerStarted","Data":"5469a0843db2ad9a228651103349ec42aeada9d65485070e194f588c5d33b897"}
Jul 08 02:40:15.630174 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.630144    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl" event={"ID":"bff0db38-2156-4511-afe0-0786c5200158","Type":"ContainerStarted","Data":"cf024bac6e2e24398688a08c29f88ecfba4e980834f632c3e84744a224c1c8c5"}
Jul 08 02:40:15.631433 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.631390    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerStarted","Data":"18a05402eb87a2103b8bbffb7445117005fc5d804a8881ecac537657f73e72c7"}
Jul 08 02:40:15.633143 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.633115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"520f868e54d9ce3ac5e8ced43b644c73995b9ae2778b2f3dc9f7d09c5832c29f"}
Jul 08 02:40:15.634949 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.634904    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5fvmr" event={"ID":"488bd2fd-661e-4813-9b10-29d42a01e530","Type":"ContainerStarted","Data":"a477349f3d5cd93804492f3a059f5f0dd4615720ca3e7c82b37003bb6230bfb9"}
Jul 08 02:40:15.636182 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.636156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p259t" event={"ID":"b4647bc1-61bf-4ef6-9547-559ff6f3ec49","Type":"ContainerStarted","Data":"14479a461cdffbf810038616944a2efa423b77588da4f412d00e4a3ed59fda21"}
Jul 08 02:40:15.637517 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.637482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b5rqr" event={"ID":"341b33aa-f143-402d-99e0-39965d30bd04","Type":"ContainerStarted","Data":"43c8de396ecfc7cc19f2e2b85d64826c6b7619a32a94e3fa34fcb50346b16708"}
Jul 08 02:40:15.638672 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:15.638644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gxfd5" event={"ID":"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663","Type":"ContainerStarted","Data":"d0a11ab9b09e09e8900e620849b2a15c583199969e78835b45f1ba195954025f"}
Jul 08 02:40:16.182852 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.182804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:16.183060 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.182911    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:16.183060 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.182938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:16.183060 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183047    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183060    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183062    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183099    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183116    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183126    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:18.183104936 +0000 UTC m=+6.110966527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183149    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:18.183138164 +0000 UTC m=+6.110999760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:16.183258 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.183172    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:18.183154503 +0000 UTC m=+6.111016113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:16.500579 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.500394    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 02:35:14 +0000 UTC" deadline="2028-02-10 17:50:31.825771164 +0000 UTC"
Jul 08 02:40:16.500579 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.500460    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13983h10m15.325315259s"
Jul 08 02:40:16.616967 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.616930    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:16.617138 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.617095    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:16.617528 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.617507    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:16.617638 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:16.617617    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:16.649380 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:16.649338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal" event={"ID":"1fa326e2985d0d87989ce38a6f8caaeb","Type":"ContainerStarted","Data":"59eba3a02c5cd5f649dbbfb4522c4f35223910be1278c7214f874460f1fa663d"}
Jul 08 02:40:17.615019 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:17.614778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:17.615019 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:17.614952    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:17.670204 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:17.670167    2558 generic.go:356] "Generic (PLEG): container finished" podID="bb75151fa942c83c5acbdad9dea7c0bc" containerID="7e5b49058d7ad65d3133cf1091f0e067238a120b2b9d0e2a64dfade2633e10c1" exitCode=0
Jul 08 02:40:17.670376 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:17.670325    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal" event={"ID":"bb75151fa942c83c5acbdad9dea7c0bc","Type":"ContainerDied","Data":"7e5b49058d7ad65d3133cf1091f0e067238a120b2b9d0e2a64dfade2633e10c1"}
Jul 08 02:40:17.692792 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:17.692728    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-88.ec2.internal" podStartSLOduration=3.692707204 podStartE2EDuration="3.692707204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:16.669251005 +0000 UTC m=+4.597112616" watchObservedRunningTime="2026-07-08 02:40:17.692707204 +0000 UTC m=+5.620568814"
Jul 08 02:40:18.200207 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:18.200167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:18.200440 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:18.200234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:18.200440 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:18.200266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:18.200440 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200409    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:18.200609 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200473    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:22.200454502 +0000 UTC m=+10.128316110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:18.200609 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200570    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:18.200609 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200587    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:18.200609 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200599    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:18.200808 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200646    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:22.200632178 +0000 UTC m=+10.128493781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:18.200808 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200719    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:18.200808 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.200750    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:22.200739846 +0000 UTC m=+10.128601451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:18.615023 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:18.614939    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:18.615186 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.615083    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:18.615186 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:18.615132    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:18.615585 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:18.615268    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:19.614131 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:19.613587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:19.614131 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:19.613725    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:20.614330 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:20.613646    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:20.614330 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:20.613883    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:20.614330 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:20.614051    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:20.614330 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:20.614197    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:21.613622 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:21.613577    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:21.613800 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:21.613717    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:22.230670 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:22.230630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:22.230688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:22.230753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.230802    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.230802    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.230903    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.230925    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.230938    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.230910    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:30.23088557 +0000 UTC m=+18.158747177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.231020    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:30.230986297 +0000 UTC m=+18.158847898 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:22.231212 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.231045    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:30.231033224 +0000 UTC m=+18.158894824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:22.615890 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:22.615345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:22.615890 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.615453    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:22.615890 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:22.615489    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:22.615890 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:22.615631    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:23.614616 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:23.613810    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:23.614616 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:23.613960    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:24.614483 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:24.614437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:24.614656 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:24.614578    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:24.615087 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:24.614949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:24.615087 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:24.615045    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:25.613995 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:25.613962    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:25.614195 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:25.614094    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:26.614021 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:26.613929    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:26.614455 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:26.614075    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:26.614455 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:26.614130    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:26.614455 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:26.614266    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:27.614449 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:27.614413    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:27.614864 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:27.614534    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:28.613661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:28.613613    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:28.613661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:28.613645    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:28.613904 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:28.613778    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:28.613971 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:28.613947    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:29.614105 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:29.614067    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:29.614550 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:29.614193    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:30.290040 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:30.289999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:30.290081    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:30.290104    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290197    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290211    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290227    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290229    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:30.290235 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290241    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:30.290480 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290272    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:46.290253456 +0000 UTC m=+34.218115059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:30.290480 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290295    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:46.290279208 +0000 UTC m=+34.218140814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:30.290480 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.290315    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:40:46.290304711 +0000 UTC m=+34.218166300 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:30.614090 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:30.614048    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:30.614257 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:30.614112    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:30.614257 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.614216    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:30.614672 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:30.614324    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:31.614072 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:31.614031    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:31.614246 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:31.614158    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:32.614347 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:32.614318    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:32.614776 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:32.614425    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:32.614776 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:32.614467    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:32.614776 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:32.614579    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:33.614188 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:33.614154    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:33.614337 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:33.614272    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:34.614732 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.614400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:34.615577 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:34.614731    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:34.615577 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.614495    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:34.615577 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:34.614937    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:34.708013 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.707979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p259t" event={"ID":"b4647bc1-61bf-4ef6-9547-559ff6f3ec49","Type":"ContainerStarted","Data":"2e5b70f403bc93291bbe3ede7cdb16e87b2a4f79a945310f0997cb7750547de9"}
Jul 08 02:40:34.709484 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.709459    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b5rqr" event={"ID":"341b33aa-f143-402d-99e0-39965d30bd04","Type":"ContainerStarted","Data":"ee86c746b43802b4bcbccc2cf6b91846f59ab89334dd061f019926e43281bc53"}
Jul 08 02:40:34.710869 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.710824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gxfd5" event={"ID":"6bd750e4-0d2e-4de2-99f8-41b0d0c0d663","Type":"ContainerStarted","Data":"a9d6b5167f90aed1703c54671eeb7b419137bebd17d9fbc6d4425a37fb964a5c"}
Jul 08 02:40:34.712744 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.712724    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal" event={"ID":"bb75151fa942c83c5acbdad9dea7c0bc","Type":"ContainerStarted","Data":"02052a9d45e1f43274198a65d2f1b6640899029a16784c89868dd956737e6021"}
Jul 08 02:40:34.714299 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.714277    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wg2vj" event={"ID":"6f6f7f3c-3e93-4a58-94f0-16e462e437e5","Type":"ContainerStarted","Data":"248b868c1a9999e5b01d66e18d11dd8cbcc9f6dfede25db2511557a56c43c8c4"}
Jul 08 02:40:34.715619 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.715598    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tts6l" event={"ID":"30e3fc41-317a-4aa5-aa4c-c76011d1d220","Type":"ContainerStarted","Data":"a27c54cd15f3a15a8672697886067b415b38e3dc895fbf30e37d9de654dcbc65"}
Jul 08 02:40:34.717005 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.716985    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl" event={"ID":"bff0db38-2156-4511-afe0-0786c5200158","Type":"ContainerStarted","Data":"9be6f99d06b482652b1b5c179d418ff70eee70f415a57cd473ad319cfca28b32"}
Jul 08 02:40:34.718495 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.718472    2558 generic.go:356] "Generic (PLEG): container finished" podID="439740dd-c52f-4805-9f49-7f1b0086f066" containerID="4f6d871bf77a990ecbec4a7b4f229601ae54fff9d2b27e8d57b8d1fa01e435f9" exitCode=0
Jul 08 02:40:34.718571 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.718541    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerDied","Data":"4f6d871bf77a990ecbec4a7b4f229601ae54fff9d2b27e8d57b8d1fa01e435f9"}
Jul 08 02:40:34.720329 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.720314    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:40:34.720710 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.720680    2558 generic.go:356] "Generic (PLEG): container finished" podID="e4c49f8e-fe3b-4cab-af44-166d9693057c" containerID="cd91b4e39f4accb7e7c169ed50e4f66a23ea5657a12bd8e5162a2798a20e3859" exitCode=1
Jul 08 02:40:34.720865 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.720726    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"f4271863833b1b5b62560f64510847d3a3eefd4cdafaf9375a2b5f6603577e4a"}
Jul 08 02:40:34.720865 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.720800    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerDied","Data":"cd91b4e39f4accb7e7c169ed50e4f66a23ea5657a12bd8e5162a2798a20e3859"}
Jul 08 02:40:34.720944 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.720884    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"dc68abbe2861620c8701143ebf8ca14c8fda21fb2d80e65bcf7c5186e3493fa9"}
Jul 08 02:40:34.745212 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.745165    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-88.ec2.internal" podStartSLOduration=20.745147951 podStartE2EDuration="20.745147951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:34.744994301 +0000 UTC m=+22.672855911" watchObservedRunningTime="2026-07-08 02:40:34.745147951 +0000 UTC m=+22.673009560"
Jul 08 02:40:34.745647 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.745611    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p259t" podStartSLOduration=2.314900663 podStartE2EDuration="21.74559869s" totalImagesPullingTime="19.430698027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:34.727596135 +0000 UTC m=+22.655457746" watchObservedRunningTime="2026-07-08 02:40:34.74559869 +0000 UTC m=+22.673460301"
Jul 08 02:40:34.835367 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.835219    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gxfd5" podStartSLOduration=3.354968256 podStartE2EDuration="22.835201759s" totalImagesPullingTime="19.480233503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:34.809517029 +0000 UTC m=+22.737378640" watchObservedRunningTime="2026-07-08 02:40:34.835201759 +0000 UTC m=+22.763063412"
Jul 08 02:40:34.835511 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.835485    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b5rqr" podStartSLOduration=2.3130748199999998 podStartE2EDuration="21.835476352s" totalImagesPullingTime="19.522401532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:34.834697777 +0000 UTC m=+22.762559386" watchObservedRunningTime="2026-07-08 02:40:34.835476352 +0000 UTC m=+22.763337982"
Jul 08 02:40:34.867441 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.867387    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tts6l" podStartSLOduration=2.448395574 podStartE2EDuration="21.867371077s" totalImagesPullingTime="19.418975503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:34.867370565 +0000 UTC m=+22.795232174" watchObservedRunningTime="2026-07-08 02:40:34.867371077 +0000 UTC m=+22.795232699"
Jul 08 02:40:34.901384 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:34.901336    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wg2vj" podStartSLOduration=2.483998111 podStartE2EDuration="21.901314053s" totalImagesPullingTime="19.417315942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:34.9012968 +0000 UTC m=+22.829158410" watchObservedRunningTime="2026-07-08 02:40:34.901314053 +0000 UTC m=+22.829175666"
Jul 08 02:40:35.613807 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.613775    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:35.614064 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:35.613912    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:35.726977 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.726943    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:40:35.727403 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.727377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"df888396ec336fdd215b5e6b3106b06b73a43ef880ade2d9189745c66b7efc4d"}
Jul 08 02:40:35.727477 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.727467    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"ba63196b031c2d532ad5efca591a72e67ca30945855d08b347773134cd92ec9d"}
Jul 08 02:40:35.727528 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.727519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"9bbf25b38070785787476c1386e9eb8c1674a923862273003d2b0a4974f26918"}
Jul 08 02:40:35.729095 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.729065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5fvmr" event={"ID":"488bd2fd-661e-4813-9b10-29d42a01e530","Type":"ContainerStarted","Data":"2e895f42bef730ee6c7d5a0d37f96632d32bc4d2b088fbedba6693f323ebbecf"}
Jul 08 02:40:35.950363 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:35.950335    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 02:40:36.520178 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:36.520068    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T02:40:35.950358336Z","UUID":"fbbef434-0db4-409b-8fe8-b6118edc6d09","Handler":null,"Name":"","Endpoint":""}
Jul 08 02:40:36.522664 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:36.522635    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 02:40:36.522806 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:36.522682    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 02:40:36.616917 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:36.616889    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:36.617071 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:36.617010    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:36.617071 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:36.617066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:36.617215 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:36.617195    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:36.733922 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:36.733887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl" event={"ID":"bff0db38-2156-4511-afe0-0786c5200158","Type":"ContainerStarted","Data":"782ff6dd50c831f4fb8ad93aeb65c74d71f46f3130bca79b44cca4460324f323"}
Jul 08 02:40:37.614200 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.613982    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:37.614324 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:37.614220    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:37.740240 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.740213    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:40:37.740685 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.740627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"7ab8c4fc73837fea8579d187cde56938f1176d5259f2536a1362048618fdeb75"}
Jul 08 02:40:37.743113 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.743057    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl" event={"ID":"bff0db38-2156-4511-afe0-0786c5200158","Type":"ContainerStarted","Data":"5021839354dab93cf1720613aceeda7456cc70b096cfe00cabecd6e5423b2318"}
Jul 08 02:40:37.770577 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.770525    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5fvmr" podStartSLOduration=5.370604976 podStartE2EDuration="24.770507149s" totalImagesPullingTime="19.399902173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:35.763577985 +0000 UTC m=+23.691439596" watchObservedRunningTime="2026-07-08 02:40:37.770507149 +0000 UTC m=+25.698368758"
Jul 08 02:40:37.770739 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.770631    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ldrhl" podStartSLOduration=2.880799239 podStartE2EDuration="24.770626107s" totalImagesPullingTime="21.889826868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:37.770237544 +0000 UTC m=+25.698099154" watchObservedRunningTime="2026-07-08 02:40:37.770626107 +0000 UTC m=+25.698487716"
Jul 08 02:40:37.939263 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.939223    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:37.939936 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:37.939918    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:38.614142 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:38.614101    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:38.614326 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:38.614234    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:38.614326 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:38.614286    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:38.614416 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:38.614390    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:38.746215 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:38.746181    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:38.746911 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:38.746892    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wg2vj"
Jul 08 02:40:39.614197 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:39.614175    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:39.614311 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:39.614278    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:39.750623 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:39.750457    2558 generic.go:356] "Generic (PLEG): container finished" podID="439740dd-c52f-4805-9f49-7f1b0086f066" containerID="98116d259257c63039be4b434f56edbdd21fc0f19899d09921d8ede319d30c5c" exitCode=0
Jul 08 02:40:39.751328 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:39.750535    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerDied","Data":"98116d259257c63039be4b434f56edbdd21fc0f19899d09921d8ede319d30c5c"}
Jul 08 02:40:39.754235 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:39.754220    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:40:39.754598 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:39.754572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"eabd257f0979907cdcbef86eb170c64ac2aa44445f7eaba94a7306317e078b96"}
Jul 08 02:40:39.755091 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:39.755069    2558 scope.go:117] "RemoveContainer" containerID="cd91b4e39f4accb7e7c169ed50e4f66a23ea5657a12bd8e5162a2798a20e3859"
Jul 08 02:40:40.614082 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.613979    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:40.614241 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.613979    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:40.614241 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:40.614116    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:40.614241 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:40.614185    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:40.771132 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.769629    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:40:40.773077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.773020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" event={"ID":"e4c49f8e-fe3b-4cab-af44-166d9693057c","Type":"ContainerStarted","Data":"3cbc72e9bb797ab163e2d2d866fac34910ea7eac0117b33fd666c24227d9f514"}
Jul 08 02:40:40.773234 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.773120    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:40.773234 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.773175    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:40.773335 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.773318    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:40.792912 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.792885    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:40.794062 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.794037    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:40:40.804241 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:40.804194    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2" podStartSLOduration=9.266186313 podStartE2EDuration="28.804176915s" totalImagesPullingTime="19.537990602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:40.8040661 +0000 UTC m=+28.731927710" watchObservedRunningTime="2026-07-08 02:40:40.804176915 +0000 UTC m=+28.732038526"
Jul 08 02:40:41.498072 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.498028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkrtz"]
Jul 08 02:40:41.498242 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.498172    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:41.498290 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:41.498257    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:41.500829 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.500804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xkg96"]
Jul 08 02:40:41.500975 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.500942    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:41.501110 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:41.501080    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:41.501399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.501377    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rvgzh"]
Jul 08 02:40:41.501483 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.501472    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:41.501568 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:41.501550    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:41.776891 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.776860    2558 generic.go:356] "Generic (PLEG): container finished" podID="439740dd-c52f-4805-9f49-7f1b0086f066" containerID="4a5683153a04da3e196576492ed5056edf844e914892132ec1dbb4801b434502" exitCode=0
Jul 08 02:40:41.777328 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:41.776945    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerDied","Data":"4a5683153a04da3e196576492ed5056edf844e914892132ec1dbb4801b434502"}
Jul 08 02:40:42.614646 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:42.614609    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:42.614884 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:42.614700    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:42.614884 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:42.614782    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:42.614996 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:42.614895    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:42.782375 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:42.782289    2558 generic.go:356] "Generic (PLEG): container finished" podID="439740dd-c52f-4805-9f49-7f1b0086f066" containerID="a6f273ff9da76b22f27c50df3297ba809e981d5fab5b2607bd15c56ea4fdd95f" exitCode=0
Jul 08 02:40:42.782870 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:42.782381    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerDied","Data":"a6f273ff9da76b22f27c50df3297ba809e981d5fab5b2607bd15c56ea4fdd95f"}
Jul 08 02:40:43.613628 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:43.613587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:43.613814 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:43.613751    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:44.613739 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:44.613703    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:44.614378 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:44.613832    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:44.614378 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:44.613878    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:44.614378 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:44.613992    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:45.613700 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:45.613662    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:45.613909 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:45.613814    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:40:46.305827 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:46.305777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:46.305827 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:46.305834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:46.305906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.305948    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306015    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306024    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306037    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret podName:10e31942-6737-4b44-a21c-3009611e2f37 nodeName:}" failed. No retries permitted until 2026-07-08 02:41:18.306016491 +0000 UTC m=+66.233878092 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret") pod "global-pull-secret-syncer-rvgzh" (UID: "10e31942-6737-4b44-a21c-3009611e2f37") : object "kube-system"/"original-pull-secret" not registered
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306048    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306060    2558 projected.go:196] Error preparing data for projected volume kube-api-access-zqbvc for pod openshift-network-diagnostics/network-check-target-hkrtz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306087    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:41:18.306062053 +0000 UTC m=+66.233923648 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 02:40:46.306205 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.306109    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc podName:abd901db-3c37-483b-97db-35a88a67f6b2 nodeName:}" failed. No retries permitted until 2026-07-08 02:41:18.306094986 +0000 UTC m=+66.233956595 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqbvc" (UniqueName: "kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc") pod "network-check-target-hkrtz" (UID: "abd901db-3c37-483b-97db-35a88a67f6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 02:40:46.614262 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:46.614171    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:46.614759 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:46.614179    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:46.614759 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.614314    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkrtz" podUID="abd901db-3c37-483b-97db-35a88a67f6b2"
Jul 08 02:40:46.614759 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:46.614405    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvgzh" podUID="10e31942-6737-4b44-a21c-3009611e2f37"
Jul 08 02:40:47.350950 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.350913    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-88.ec2.internal" event="NodeReady"
Jul 08 02:40:47.351143 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.351062    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 02:40:47.399114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.398985    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f8hwx"]
Jul 08 02:40:47.429279 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.429240    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rrmmk"]
Jul 08 02:40:47.429451 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.429392    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.432624 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.432598    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 02:40:47.432778 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.432682    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8d2rg\""
Jul 08 02:40:47.433123 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.432875    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 02:40:47.445238 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.445134    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrmmk"]
Jul 08 02:40:47.445238 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.445172    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f8hwx"]
Jul 08 02:40:47.445427 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.445270    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.448058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.447892    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 02:40:47.448058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.447912    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 02:40:47.448058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.448048    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 02:40:47.448276 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.448081    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c7hfw\""
Jul 08 02:40:47.515128 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.515095    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.515327 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.515163    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gplq\" (UniqueName: \"kubernetes.io/projected/72c48082-e39b-4908-a3b9-07cf0759e80b-kube-api-access-4gplq\") pod \"ingress-canary-rrmmk\" (UID: \"72c48082-e39b-4908-a3b9-07cf0759e80b\") " pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.515327 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.515264    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbgjc\" (UniqueName: \"kubernetes.io/projected/d011ddb5-0557-4f54-b4a5-22abc3e9798c-kube-api-access-pbgjc\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.515327 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.515299    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72c48082-e39b-4908-a3b9-07cf0759e80b-cert\") pod \"ingress-canary-rrmmk\" (UID: \"72c48082-e39b-4908-a3b9-07cf0759e80b\") " pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.515482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.515332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d011ddb5-0557-4f54-b4a5-22abc3e9798c-tmp-dir\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.515482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.515383    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d011ddb5-0557-4f54-b4a5-22abc3e9798c-config-volume\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.613795 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.613713    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616374    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gplq\" (UniqueName: \"kubernetes.io/projected/72c48082-e39b-4908-a3b9-07cf0759e80b-kube-api-access-4gplq\") pod \"ingress-canary-rrmmk\" (UID: \"72c48082-e39b-4908-a3b9-07cf0759e80b\") " pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbgjc\" (UniqueName: \"kubernetes.io/projected/d011ddb5-0557-4f54-b4a5-22abc3e9798c-kube-api-access-pbgjc\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72c48082-e39b-4908-a3b9-07cf0759e80b-cert\") pod \"ingress-canary-rrmmk\" (UID: \"72c48082-e39b-4908-a3b9-07cf0759e80b\") " pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d011ddb5-0557-4f54-b4a5-22abc3e9798c-tmp-dir\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d011ddb5-0557-4f54-b4a5-22abc3e9798c-config-volume\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616572    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:47.616665    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:47.616729    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:40:48.116709264 +0000 UTC m=+36.044570867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616900    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 02:40:47.617077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.616965    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n5djs\""
Jul 08 02:40:47.617943 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.617110    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d011ddb5-0557-4f54-b4a5-22abc3e9798c-tmp-dir\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.617943 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.617312    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d011ddb5-0557-4f54-b4a5-22abc3e9798c-config-volume\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.622436 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.622226    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72c48082-e39b-4908-a3b9-07cf0759e80b-cert\") pod \"ingress-canary-rrmmk\" (UID: \"72c48082-e39b-4908-a3b9-07cf0759e80b\") " pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.628516 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.628487    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gplq\" (UniqueName: \"kubernetes.io/projected/72c48082-e39b-4908-a3b9-07cf0759e80b-kube-api-access-4gplq\") pod \"ingress-canary-rrmmk\" (UID: \"72c48082-e39b-4908-a3b9-07cf0759e80b\") " pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.628796 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.628743    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbgjc\" (UniqueName: \"kubernetes.io/projected/d011ddb5-0557-4f54-b4a5-22abc3e9798c-kube-api-access-pbgjc\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:47.755133 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.755098    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrmmk"
Jul 08 02:40:47.948780 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:47.948574    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrmmk"]
Jul 08 02:40:48.017621 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:40:48.017581    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c48082_e39b_4908_a3b9_07cf0759e80b.slice/crio-7329e39caff3858a7b89671eae5b9c2418dcc670553052fa42db506d9e86e716 WatchSource:0}: Error finding container 7329e39caff3858a7b89671eae5b9c2418dcc670553052fa42db506d9e86e716: Status 404 returned error can't find the container with id 7329e39caff3858a7b89671eae5b9c2418dcc670553052fa42db506d9e86e716
Jul 08 02:40:48.121241 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.121163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:48.121388 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:48.121274    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:40:48.121388 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:48.121324    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:40:49.121311304 +0000 UTC m=+37.049172891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:40:48.614360 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.614327    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:40:48.614806 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.614789    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:40:48.617338 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.617313    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 02:40:48.617741 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.617365    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 02:40:48.618371 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.618352    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 02:40:48.618490 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.618351    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d2c2d\""
Jul 08 02:40:48.798269 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.798226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrmmk" event={"ID":"72c48082-e39b-4908-a3b9-07cf0759e80b","Type":"ContainerStarted","Data":"7329e39caff3858a7b89671eae5b9c2418dcc670553052fa42db506d9e86e716"}
Jul 08 02:40:48.801565 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.801536    2558 generic.go:356] "Generic (PLEG): container finished" podID="439740dd-c52f-4805-9f49-7f1b0086f066" containerID="f4f38d6150d8f7858e6fd5615288c0e52fd112db955eba7a24feec8cf4e7b744" exitCode=0
Jul 08 02:40:48.801728 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:48.801596    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerDied","Data":"f4f38d6150d8f7858e6fd5615288c0e52fd112db955eba7a24feec8cf4e7b744"}
Jul 08 02:40:49.130896 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:49.130796    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:49.131050 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:49.130981    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:40:49.131109 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:49.131063    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:40:51.131045334 +0000 UTC m=+39.058906946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:40:49.807886 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:49.807827    2558 generic.go:356] "Generic (PLEG): container finished" podID="439740dd-c52f-4805-9f49-7f1b0086f066" containerID="9d2ec5a7f34fc6cfef3b2b3d318cae6567f97fea9882ab947fafb47b70f3cf57" exitCode=0
Jul 08 02:40:49.808358 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:49.807887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerDied","Data":"9d2ec5a7f34fc6cfef3b2b3d318cae6567f97fea9882ab947fafb47b70f3cf57"}
Jul 08 02:40:50.813720 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:50.813478    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" event={"ID":"439740dd-c52f-4805-9f49-7f1b0086f066","Type":"ContainerStarted","Data":"dc09f489b122c49d4cd1de125d3c38d8ec271e8eb92f73f498778ae694f9a0f9"}
Jul 08 02:40:50.815051 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:50.815027    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrmmk" event={"ID":"72c48082-e39b-4908-a3b9-07cf0759e80b","Type":"ContainerStarted","Data":"45441fa1f743e0bc7101c5f9e163694671207d3766f837e39b2a0a8cce63641a"}
Jul 08 02:40:50.837778 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:50.837736    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7sd9k" podStartSLOduration=7.74262873 podStartE2EDuration="38.837722596s" totalImagesPullingTime="31.095093866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:50.836903856 +0000 UTC m=+38.764765465" watchObservedRunningTime="2026-07-08 02:40:50.837722596 +0000 UTC m=+38.765584206"
Jul 08 02:40:50.853223 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:50.853176    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rrmmk" podStartSLOduration=1.396568037 podStartE2EDuration="3.853161948s" totalImagesPullingTime="2.456593911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:40:50.852972751 +0000 UTC m=+38.780834363" watchObservedRunningTime="2026-07-08 02:40:50.853161948 +0000 UTC m=+38.781023557"
Jul 08 02:40:51.145013 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:51.144936    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:51.145136 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:51.145060    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:40:51.145136 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:51.145127    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:40:55.14511384 +0000 UTC m=+43.072975428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:40:55.172934 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:40:55.172889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:40:55.173597 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:55.173023    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:40:55.173597 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:40:55.173088    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:41:03.173071106 +0000 UTC m=+51.100932713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:41:03.230643 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:03.230606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:41:03.231131 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:03.230715    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:41:03.231131 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:03.230771    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:41:19.230757655 +0000 UTC m=+67.158619257 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:41:12.793640 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:12.793612    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzzq2"
Jul 08 02:41:18.335293 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.335252    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:41:18.335794 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.335318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:41:18.335794 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.335344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:41:18.338159 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.338135    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 02:41:18.338211 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.338167    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 02:41:18.338211 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.338182    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 02:41:18.346314 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:18.346294    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 02:41:18.346435 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:18.346360    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:42:22.346343738 +0000 UTC m=+130.274205339 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : secret "metrics-daemon-secret" not found
Jul 08 02:41:18.348640 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.348619    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 02:41:18.350308 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.350281    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10e31942-6737-4b44-a21c-3009611e2f37-original-pull-secret\") pod \"global-pull-secret-syncer-rvgzh\" (UID: \"10e31942-6737-4b44-a21c-3009611e2f37\") " pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:41:18.359790 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.359765    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqbvc\" (UniqueName: \"kubernetes.io/projected/abd901db-3c37-483b-97db-35a88a67f6b2-kube-api-access-zqbvc\") pod \"network-check-target-hkrtz\" (UID: \"abd901db-3c37-483b-97db-35a88a67f6b2\") " pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:41:18.628557 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.628478    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvgzh"
Jul 08 02:41:18.640500 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.640472    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d2c2d\""
Jul 08 02:41:18.648465 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.648449    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:41:18.796000 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.795969    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rvgzh"]
Jul 08 02:41:18.800059 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:41:18.800031    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e31942_6737_4b44_a21c_3009611e2f37.slice/crio-1dc2382c9866458af46a16d0a2d70c3b9620368170af13678d7d5f99d26e6fbd WatchSource:0}: Error finding container 1dc2382c9866458af46a16d0a2d70c3b9620368170af13678d7d5f99d26e6fbd: Status 404 returned error can't find the container with id 1dc2382c9866458af46a16d0a2d70c3b9620368170af13678d7d5f99d26e6fbd
Jul 08 02:41:18.814306 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.814268    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkrtz"]
Jul 08 02:41:18.816970 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:41:18.816949    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd901db_3c37_483b_97db_35a88a67f6b2.slice/crio-b858de0aa8c53db68fea0c68e1a506766b45b0924e8cc152f59e013322554653 WatchSource:0}: Error finding container b858de0aa8c53db68fea0c68e1a506766b45b0924e8cc152f59e013322554653: Status 404 returned error can't find the container with id b858de0aa8c53db68fea0c68e1a506766b45b0924e8cc152f59e013322554653
Jul 08 02:41:18.884823 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.884737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkrtz" event={"ID":"abd901db-3c37-483b-97db-35a88a67f6b2","Type":"ContainerStarted","Data":"b858de0aa8c53db68fea0c68e1a506766b45b0924e8cc152f59e013322554653"}
Jul 08 02:41:18.885787 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:18.885766    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rvgzh" event={"ID":"10e31942-6737-4b44-a21c-3009611e2f37","Type":"ContainerStarted","Data":"1dc2382c9866458af46a16d0a2d70c3b9620368170af13678d7d5f99d26e6fbd"}
Jul 08 02:41:19.242282 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:19.242240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:41:19.242462 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:19.242377    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:41:19.242462 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:19.242455    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:41:51.242435719 +0000 UTC m=+99.170297309 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:41:21.896778 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:21.896736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkrtz" event={"ID":"abd901db-3c37-483b-97db-35a88a67f6b2","Type":"ContainerStarted","Data":"cbee7097d62005f41df60a996095cca6f02271005fbb5d8c3c6d6d872c416f86"}
Jul 08 02:41:21.897190 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:21.896893    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:41:34.500187 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:34.500065    2558 controller.go:195] "Failed to update lease" err="Put \"https://a4536e496c57f484fb86c43d48887d39-dd8d149628a69fcd.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-129-88.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 08 02:41:44.501215 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:44.501158    2558 controller.go:195] "Failed to update lease" err="Put \"https://a4536e496c57f484fb86c43d48887d39-dd8d149628a69fcd.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-129-88.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 08 02:41:49.122372 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.122332    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"]
Jul 08 02:41:49.125999 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.125976    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"]
Jul 08 02:41:49.125999 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.126007    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"]
Jul 08 02:41:49.126192 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.126108    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.128539 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.128521    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"]
Jul 08 02:41:49.128618 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.128606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.131451 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.131428    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 08 02:41:49.131554 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.131450    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 08 02:41:49.131554 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.131478    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 08 02:41:49.131708 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.131561    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 08 02:41:49.132584 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.132566    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 08 02:41:49.132694 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.132568    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 08 02:41:49.136278 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.136259    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 08 02:41:49.136357 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.136297    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 08 02:41:49.142653 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.142631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/44c1ae27-f9df-42bf-8297-a723f60762c3-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.142741 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.142680    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m6lt\" (UniqueName: \"kubernetes.io/projected/44c1ae27-f9df-42bf-8297-a723f60762c3-kube-api-access-4m6lt\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.142741 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.142702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44c1ae27-f9df-42bf-8297-a723f60762c3-tmp\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.145148 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.145110    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hkrtz" podStartSLOduration=94.294071677 podStartE2EDuration="1m37.145098123s" totalImagesPullingTime="2.851026446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:41:49.143792735 +0000 UTC m=+97.071654346" watchObservedRunningTime="2026-07-08 02:41:49.145098123 +0000 UTC m=+97.072959733"
Jul 08 02:41:49.243137 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-ca\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.243137 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243132    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243161    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m6lt\" (UniqueName: \"kubernetes.io/projected/44c1ae27-f9df-42bf-8297-a723f60762c3-kube-api-access-4m6lt\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44c1ae27-f9df-42bf-8297-a723f60762c3-tmp\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243227    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbbjk\" (UniqueName: \"kubernetes.io/projected/e300e301-72d3-4dde-b11b-78ea112e2f9d-kube-api-access-wbbjk\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/44c1ae27-f9df-42bf-8297-a723f60762c3-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-hub\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.243337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e300e301-72d3-4dde-b11b-78ea112e2f9d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.243696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.243675    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44c1ae27-f9df-42bf-8297-a723f60762c3-tmp\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.246048 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.246025    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/44c1ae27-f9df-42bf-8297-a723f60762c3-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.251314 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.251290    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m6lt\" (UniqueName: \"kubernetes.io/projected/44c1ae27-f9df-42bf-8297-a723f60762c3-kube-api-access-4m6lt\") pod \"klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2\" (UID: \"44c1ae27-f9df-42bf-8297-a723f60762c3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.343964 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.343926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.344176 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.343972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-hub\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.344176 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.344015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e300e301-72d3-4dde-b11b-78ea112e2f9d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.344176 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.344073    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-ca\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.344321 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.344195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.344321 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.344249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbbjk\" (UniqueName: \"kubernetes.io/projected/e300e301-72d3-4dde-b11b-78ea112e2f9d-kube-api-access-wbbjk\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.344863 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.344816    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e300e301-72d3-4dde-b11b-78ea112e2f9d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.346573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.346540    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-ca\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.346755 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.346711    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-hub\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.347075 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.347060    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.347144 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.347122    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e300e301-72d3-4dde-b11b-78ea112e2f9d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.353393 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.353370    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbbjk\" (UniqueName: \"kubernetes.io/projected/e300e301-72d3-4dde-b11b-78ea112e2f9d-kube-api-access-wbbjk\") pod \"cluster-proxy-proxy-agent-675944bb65-qjqjv\" (UID: \"e300e301-72d3-4dde-b11b-78ea112e2f9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.436647 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.436607    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:49.461762 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.461733    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:41:49.590092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.590054    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"]
Jul 08 02:41:49.595547 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:41:49.595511    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c1ae27_f9df_42bf_8297_a723f60762c3.slice/crio-8e55003e720247f00badc5191911f3ef17bcd49af187bd8e618e4d49dbf456c0 WatchSource:0}: Error finding container 8e55003e720247f00badc5191911f3ef17bcd49af187bd8e618e4d49dbf456c0: Status 404 returned error can't find the container with id 8e55003e720247f00badc5191911f3ef17bcd49af187bd8e618e4d49dbf456c0
Jul 08 02:41:49.626083 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.626053    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"]
Jul 08 02:41:49.630262 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:41:49.630236    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode300e301_72d3_4dde_b11b_78ea112e2f9d.slice/crio-054572e2a6b8ee613e62e40b0bc2006855cebd59b405a90c67de6a7d296d464e WatchSource:0}: Error finding container 054572e2a6b8ee613e62e40b0bc2006855cebd59b405a90c67de6a7d296d464e: Status 404 returned error can't find the container with id 054572e2a6b8ee613e62e40b0bc2006855cebd59b405a90c67de6a7d296d464e
Jul 08 02:41:49.964989 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.964949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2" event={"ID":"44c1ae27-f9df-42bf-8297-a723f60762c3","Type":"ContainerStarted","Data":"8e55003e720247f00badc5191911f3ef17bcd49af187bd8e618e4d49dbf456c0"}
Jul 08 02:41:49.965975 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:49.965951    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" event={"ID":"e300e301-72d3-4dde-b11b-78ea112e2f9d","Type":"ContainerStarted","Data":"054572e2a6b8ee613e62e40b0bc2006855cebd59b405a90c67de6a7d296d464e"}
Jul 08 02:41:51.260011 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:51.259968    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:41:51.260467 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:51.260159    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:41:51.260467 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:41:51.260233    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:42:55.260209505 +0000 UTC m=+163.188071096 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:41:52.901592 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:52.901529    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hkrtz"
Jul 08 02:41:54.984891 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:54.984619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2" event={"ID":"44c1ae27-f9df-42bf-8297-a723f60762c3","Type":"ContainerStarted","Data":"529d730d349dbfaf17fc934e7ff22eab0bf47bc3b1550919698c13c72bc7091b"}
Jul 08 02:41:54.984891 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:54.984813    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:54.986260 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:54.986211    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2" podUID="44c1ae27-f9df-42bf-8297-a723f60762c3" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Jul 08 02:41:54.986695 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:54.986649    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" event={"ID":"e300e301-72d3-4dde-b11b-78ea112e2f9d","Type":"ContainerStarted","Data":"c5056d8064da55a966847fc31fcd6d4daf514f3fded22f24088600017390f27f"}
Jul 08 02:41:55.995428 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:55.995392    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" event={"ID":"e300e301-72d3-4dde-b11b-78ea112e2f9d","Type":"ContainerStarted","Data":"2bef56525a49cf39c051b6025e94ab65f549e44843749e1abd8bd329a1bc86ac"}
Jul 08 02:41:55.995428 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:55.995441    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" event={"ID":"e300e301-72d3-4dde-b11b-78ea112e2f9d","Type":"ContainerStarted","Data":"1279f6c8164f274c0267a1f8f4f020bef3d27bd45f115518565e87e25ba1518c"}
Jul 08 02:41:55.996473 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:55.996446    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:41:56.025895 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:56.025829    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2" podStartSLOduration=20.708867662 podStartE2EDuration="26.025815126s" totalImagesPullingTime="5.316947464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:41:55.010936458 +0000 UTC m=+102.938798069" watchObservedRunningTime="2026-07-08 02:41:56.025815126 +0000 UTC m=+103.953676736"
Jul 08 02:41:56.026062 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:41:56.026041    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" podStartSLOduration=20.745489366 podStartE2EDuration="26.026035801s" totalImagesPullingTime="5.280546435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:41:56.024387373 +0000 UTC m=+103.952248985" watchObservedRunningTime="2026-07-08 02:41:56.026035801 +0000 UTC m=+103.953897402"
Jul 08 02:42:02.014495 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:42:02.014455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rvgzh" event={"ID":"10e31942-6737-4b44-a21c-3009611e2f37","Type":"ContainerStarted","Data":"67c9279039b406bd0758c4f3d53531cd586960aabf11a6ab98c038f426726d51"}
Jul 08 02:42:02.045826 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:42:02.045772    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rvgzh" podStartSLOduration=66.832186315 podStartE2EDuration="1m49.045758569s" totalImagesPullingTime="42.213572254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:42:02.045578645 +0000 UTC m=+109.973440255" watchObservedRunningTime="2026-07-08 02:42:02.045758569 +0000 UTC m=+109.973620179"
Jul 08 02:42:22.393382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:42:22.393348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:42:22.393895 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:42:22.393463    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 02:42:22.393895 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:42:22.393515    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs podName:0cb017c8-2eee-465b-9639-575474c42599 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:24.393502322 +0000 UTC m=+252.321363910 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs") pod "network-metrics-daemon-xkg96" (UID: "0cb017c8-2eee-465b-9639-575474c42599") : secret "metrics-daemon-secret" not found
Jul 08 02:42:50.440453 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:42:50.440396    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f8hwx" podUID="d011ddb5-0557-4f54-b4a5-22abc3e9798c"
Jul 08 02:42:50.630236 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:42:50.630193    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xkg96" podUID="0cb017c8-2eee-465b-9639-575474c42599"
Jul 08 02:42:51.155248 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:42:51.155222    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:42:55.329324 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:42:55.329276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:42:55.329731 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:42:55.329398    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 02:42:55.329731 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:42:55.329467    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls podName:d011ddb5-0557-4f54-b4a5-22abc3e9798c nodeName:}" failed. No retries permitted until 2026-07-08 02:44:57.32945157 +0000 UTC m=+285.257313157 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls") pod "dns-default-f8hwx" (UID: "d011ddb5-0557-4f54-b4a5-22abc3e9798c") : secret "dns-default-metrics-tls" not found
Jul 08 02:43:03.613828 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:03.613721    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:43:55.336776 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:55.336682    2558 generic.go:356] "Generic (PLEG): container finished" podID="44c1ae27-f9df-42bf-8297-a723f60762c3" containerID="529d730d349dbfaf17fc934e7ff22eab0bf47bc3b1550919698c13c72bc7091b" exitCode=1
Jul 08 02:43:55.336776 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:55.336739    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2" event={"ID":"44c1ae27-f9df-42bf-8297-a723f60762c3","Type":"ContainerDied","Data":"529d730d349dbfaf17fc934e7ff22eab0bf47bc3b1550919698c13c72bc7091b"}
Jul 08 02:43:55.337433 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:55.337162    2558 scope.go:117] "RemoveContainer" containerID="529d730d349dbfaf17fc934e7ff22eab0bf47bc3b1550919698c13c72bc7091b"
Jul 08 02:43:55.996508 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:55.996455    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:43:56.341114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:56.341027    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2" event={"ID":"44c1ae27-f9df-42bf-8297-a723f60762c3","Type":"ContainerStarted","Data":"bb6ade13d0f63651bb474a8b49e5e954bd03fb1879338fc5357f8b4eb2b4180b"}
Jul 08 02:43:56.341479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:56.341242    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:43:56.341890 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:43:56.341872    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfb6cb6d8-cjrf2"
Jul 08 02:44:06.992374 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.992339    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jl2lq"]
Jul 08 02:44:06.994651 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.994630    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:06.997431 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.997404    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 02:44:06.998479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.998456    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 02:44:06.998479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.998477    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 02:44:06.998649 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.998511    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 02:44:06.998649 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:06.998572    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6xd4b\""
Jul 08 02:44:07.006692 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.006675    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jl2lq"]
Jul 08 02:44:07.128801 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.128760    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.129030 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.128807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d16de454-b9dc-4872-a160-26e4e0db8a20-crio-socket\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.129030 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.128859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5fc\" (UniqueName: \"kubernetes.io/projected/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-api-access-hm5fc\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.129030 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.128899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d16de454-b9dc-4872-a160-26e4e0db8a20-data-volume\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.129030 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.128925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230164 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d16de454-b9dc-4872-a160-26e4e0db8a20-data-volume\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230164 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230319 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230319 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d16de454-b9dc-4872-a160-26e4e0db8a20-crio-socket\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230319 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230292    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5fc\" (UniqueName: \"kubernetes.io/projected/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-api-access-hm5fc\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230406 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.230350    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 02:44:07.230406 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230373    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d16de454-b9dc-4872-a160-26e4e0db8a20-crio-socket\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.230406 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.230385    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:07.230491 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.230431    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:07.730414144 +0000 UTC m=+235.658275732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : configmap "kube-rbac-proxy" not found
Jul 08 02:44:07.230491 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.230451    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:07.730438121 +0000 UTC m=+235.658299708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:07.230565 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.230527    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d16de454-b9dc-4872-a160-26e4e0db8a20-data-volume\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.244772 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.244699    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5fc\" (UniqueName: \"kubernetes.io/projected/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-api-access-hm5fc\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.734083 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.734050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.734290 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.734186    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 02:44:07.734290 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:07.734261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:07.734401 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.734355    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:08.734327238 +0000 UTC m=+236.662188874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : configmap "kube-rbac-proxy" not found
Jul 08 02:44:07.734401 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.734391    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:07.734486 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:07.734431    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:08.734421924 +0000 UTC m=+236.662283511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:08.739354 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:08.739306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:08.739752 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:08.739378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:08.739752 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:08.739445    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 02:44:08.739752 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:08.739469    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:08.739752 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:08.739525    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:10.739507872 +0000 UTC m=+238.667369461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : configmap "kube-rbac-proxy" not found
Jul 08 02:44:08.739752 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:08.739541    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:10.739535096 +0000 UTC m=+238.667396685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:08.979279 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:08.979251    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p259t_b4647bc1-61bf-4ef6-9547-559ff6f3ec49/dns-node-resolver/0.log"
Jul 08 02:44:09.976142 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:09.976115    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tts6l_30e3fc41-317a-4aa5-aa4c-c76011d1d220/node-ca/0.log"
Jul 08 02:44:10.753152 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:10.753116    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:10.753326 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:10.753165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:10.753326 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:10.753242    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 02:44:10.753326 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:10.753273    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:10.753326 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:10.753292    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:14.753279729 +0000 UTC m=+242.681141317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : configmap "kube-rbac-proxy" not found
Jul 08 02:44:10.753326 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:10.753322    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:14.753308381 +0000 UTC m=+242.681169969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:10.976015 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:10.975979    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrmmk_72c48082-e39b-4908-a3b9-07cf0759e80b/serve-healthcheck-canary/0.log"
Jul 08 02:44:14.779953 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:14.779897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:14.779953 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:14.779963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:14.780374 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:14.780057    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:14.780374 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:14.780151    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls podName:d16de454-b9dc-4872-a160-26e4e0db8a20 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:22.780131169 +0000 UTC m=+250.707992777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jl2lq" (UID: "d16de454-b9dc-4872-a160-26e4e0db8a20") : secret "insights-runtime-extractor-tls" not found
Jul 08 02:44:14.780537 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:14.780514    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d16de454-b9dc-4872-a160-26e4e0db8a20-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:22.839912 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:22.839868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:22.842346 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:22.842325    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d16de454-b9dc-4872-a160-26e4e0db8a20-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jl2lq\" (UID: \"d16de454-b9dc-4872-a160-26e4e0db8a20\") " pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:22.907041 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:22.907006    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6xd4b\""
Jul 08 02:44:22.914197 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:22.914163    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jl2lq"
Jul 08 02:44:23.041236 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:23.041200    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jl2lq"]
Jul 08 02:44:23.045667 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:44:23.045643    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd16de454_b9dc_4872_a160_26e4e0db8a20.slice/crio-da5be9ac8b633fd2813379262dae992ddf099ab5d2fd96411089bc7b6abb1c77 WatchSource:0}: Error finding container da5be9ac8b633fd2813379262dae992ddf099ab5d2fd96411089bc7b6abb1c77: Status 404 returned error can't find the container with id da5be9ac8b633fd2813379262dae992ddf099ab5d2fd96411089bc7b6abb1c77
Jul 08 02:44:23.422712 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:23.422673    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jl2lq" event={"ID":"d16de454-b9dc-4872-a160-26e4e0db8a20","Type":"ContainerStarted","Data":"da5be9ac8b633fd2813379262dae992ddf099ab5d2fd96411089bc7b6abb1c77"}
Jul 08 02:44:24.449310 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:24.449272    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:44:24.452288 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:24.452262    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb017c8-2eee-465b-9639-575474c42599-metrics-certs\") pod \"network-metrics-daemon-xkg96\" (UID: \"0cb017c8-2eee-465b-9639-575474c42599\") " pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:44:24.618930 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:24.618903    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n5djs\""
Jul 08 02:44:24.626138 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:24.626111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xkg96"
Jul 08 02:44:26.805127 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:26.805092    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xkg96"]
Jul 08 02:44:26.810674 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:44:26.810640    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb017c8_2eee_465b_9639_575474c42599.slice/crio-4c781d0285bc6f004ecb3e2610750dc40e6988f4cd0855395f4247c273298bd4 WatchSource:0}: Error finding container 4c781d0285bc6f004ecb3e2610750dc40e6988f4cd0855395f4247c273298bd4: Status 404 returned error can't find the container with id 4c781d0285bc6f004ecb3e2610750dc40e6988f4cd0855395f4247c273298bd4
Jul 08 02:44:27.439438 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:27.439400    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jl2lq" event={"ID":"d16de454-b9dc-4872-a160-26e4e0db8a20","Type":"ContainerStarted","Data":"8e76725f5463791982ca9f2255903da1d6f3633c9fe0992838c714f3aab53b65"}
Jul 08 02:44:27.439709 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:27.439467    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jl2lq" event={"ID":"d16de454-b9dc-4872-a160-26e4e0db8a20","Type":"ContainerStarted","Data":"d687539b725bd2769b1d4abdf7632a85e81660ac6a7720ce9c91be3dae72604e"}
Jul 08 02:44:27.441061 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:27.440949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xkg96" event={"ID":"0cb017c8-2eee-465b-9639-575474c42599","Type":"ContainerStarted","Data":"4c781d0285bc6f004ecb3e2610750dc40e6988f4cd0855395f4247c273298bd4"}
Jul 08 02:44:28.446639 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:28.446599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xkg96" event={"ID":"0cb017c8-2eee-465b-9639-575474c42599","Type":"ContainerStarted","Data":"d61f2c31101010d1bbd217563b241c96e70db8954156d7abde2e92d1d301c086"}
Jul 08 02:44:28.447167 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:28.446665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xkg96" event={"ID":"0cb017c8-2eee-465b-9639-575474c42599","Type":"ContainerStarted","Data":"d968b68ad978665360481871f7d6211a40fdfd57bb4a1a5e5447f6b36cda8050"}
Jul 08 02:44:28.466527 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:28.466466    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xkg96" podStartSLOduration=254.571137864 podStartE2EDuration="4m15.466445873s" totalImagesPullingTime="895.308009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:44:28.465142837 +0000 UTC m=+256.393004446" watchObservedRunningTime="2026-07-08 02:44:28.466445873 +0000 UTC m=+256.394307484"
Jul 08 02:44:29.463719 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:29.463676    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" podUID="e300e301-72d3-4dde-b11b-78ea112e2f9d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 08 02:44:30.455040 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.455001    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jl2lq" event={"ID":"d16de454-b9dc-4872-a160-26e4e0db8a20","Type":"ContainerStarted","Data":"0be6495d2afcfee731112284b336ef991645e9283e26eb39958c8cba8c5243d9"}
Jul 08 02:44:30.472774 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.472726    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jl2lq" podStartSLOduration=17.829631513 podStartE2EDuration="24.472711711s" totalImagesPullingTime="6.643080198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:44:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:44:30.472271568 +0000 UTC m=+258.400133179" watchObservedRunningTime="2026-07-08 02:44:30.472711711 +0000 UTC m=+258.400573321"
Jul 08 02:44:30.781971 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.781895    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b74d44cc5-n5rrr"]
Jul 08 02:44:30.784142 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.784121    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.787715 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.787625    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 02:44:30.787873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.787717    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 02:44:30.787873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.787632    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tthw\""
Jul 08 02:44:30.790926 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.790907    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 02:44:30.804785 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.804759    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 02:44:30.806537 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.806514    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b74d44cc5-n5rrr"]
Jul 08 02:44:30.898977 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.898934    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14b3f5ab-1cb9-45c6-a468-8f539636528e-trusted-ca\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899003    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl9rv\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-kube-api-access-gl9rv\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/14b3f5ab-1cb9-45c6-a468-8f539636528e-image-registry-private-configuration\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-registry-tls\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-bound-sa-token\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899204    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/14b3f5ab-1cb9-45c6-a468-8f539636528e-registry-certificates\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/14b3f5ab-1cb9-45c6-a468-8f539636528e-ca-trust-extracted\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:30.899382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:30.899288    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/14b3f5ab-1cb9-45c6-a468-8f539636528e-installation-pull-secrets\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000177 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-registry-tls\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000177 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-bound-sa-token\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000420 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000200    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/14b3f5ab-1cb9-45c6-a468-8f539636528e-registry-certificates\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000420 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/14b3f5ab-1cb9-45c6-a468-8f539636528e-ca-trust-extracted\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000420 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000350    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/14b3f5ab-1cb9-45c6-a468-8f539636528e-installation-pull-secrets\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000420 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000412    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14b3f5ab-1cb9-45c6-a468-8f539636528e-trusted-ca\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000612 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl9rv\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-kube-api-access-gl9rv\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000612 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000551    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/14b3f5ab-1cb9-45c6-a468-8f539636528e-image-registry-private-configuration\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.000708 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.000631    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/14b3f5ab-1cb9-45c6-a468-8f539636528e-ca-trust-extracted\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.001169 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.001131    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/14b3f5ab-1cb9-45c6-a468-8f539636528e-registry-certificates\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.001738 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.001713    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14b3f5ab-1cb9-45c6-a468-8f539636528e-trusted-ca\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.002925 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.002898    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/14b3f5ab-1cb9-45c6-a468-8f539636528e-image-registry-private-configuration\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.003022 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.002971    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-registry-tls\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.003135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.003109    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/14b3f5ab-1cb9-45c6-a468-8f539636528e-installation-pull-secrets\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.009434 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.009412    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-bound-sa-token\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.009739 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.009721    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl9rv\" (UniqueName: \"kubernetes.io/projected/14b3f5ab-1cb9-45c6-a468-8f539636528e-kube-api-access-gl9rv\") pod \"image-registry-5b74d44cc5-n5rrr\" (UID: \"14b3f5ab-1cb9-45c6-a468-8f539636528e\") " pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.099385 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.099293    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.227333 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.227301    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b74d44cc5-n5rrr"]
Jul 08 02:44:31.231125 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:44:31.231101    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b3f5ab_1cb9_45c6_a468_8f539636528e.slice/crio-346ab51ad667090d436fe62069d65689f4223a55721e7b571499fa2535830315 WatchSource:0}: Error finding container 346ab51ad667090d436fe62069d65689f4223a55721e7b571499fa2535830315: Status 404 returned error can't find the container with id 346ab51ad667090d436fe62069d65689f4223a55721e7b571499fa2535830315
Jul 08 02:44:31.460908 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.460877    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" event={"ID":"14b3f5ab-1cb9-45c6-a468-8f539636528e","Type":"ContainerStarted","Data":"5effdd724467f955232a0588a66b51c9b428d186fd6beb0cfd3f26f2ad0978c7"}
Jul 08 02:44:31.461097 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.460930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" event={"ID":"14b3f5ab-1cb9-45c6-a468-8f539636528e","Type":"ContainerStarted","Data":"346ab51ad667090d436fe62069d65689f4223a55721e7b571499fa2535830315"}
Jul 08 02:44:31.461097 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.460954    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:44:31.485626 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:31.485575    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podStartSLOduration=1.485560118 podStartE2EDuration="1.485560118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:44:31.485181231 +0000 UTC m=+259.413042843" watchObservedRunningTime="2026-07-08 02:44:31.485560118 +0000 UTC m=+259.413421724"
Jul 08 02:44:39.463662 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:39.463569    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" podUID="e300e301-72d3-4dde-b11b-78ea112e2f9d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 08 02:44:44.140927 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.140894    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74844f778b-nq7r9"]
Jul 08 02:44:44.147177 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.147150    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.149876 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.149854    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 02:44:44.150783 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.150761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 02:44:44.150923 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.150769    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 02:44:44.150923 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.150918    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 02:44:44.151036 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.150927    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ws27b\""
Jul 08 02:44:44.151036 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.150927    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 02:44:44.151141 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.151051    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 02:44:44.151855 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.151824    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 02:44:44.158245 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.158222    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 02:44:44.158343 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.158231    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74844f778b-nq7r9"]
Jul 08 02:44:44.292275 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292231    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-console-config\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.292275 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292274    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-serving-cert\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.292513 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292300    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-service-ca\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.292513 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292357    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjlz\" (UniqueName: \"kubernetes.io/projected/558988cf-59de-45e5-a7b6-0ad656640e81-kube-api-access-cdjlz\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.292513 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-trusted-ca-bundle\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.292513 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292442    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-oauth-config\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.292513 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.292477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-oauth-serving-cert\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.393758 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.393673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-console-config\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.393758 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.393723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-serving-cert\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.393942 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.393877    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-service-ca\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394008 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.393940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjlz\" (UniqueName: \"kubernetes.io/projected/558988cf-59de-45e5-a7b6-0ad656640e81-kube-api-access-cdjlz\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394008 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.393979    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-trusted-ca-bundle\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394104 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.394025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-oauth-config\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394104 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.394079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-oauth-serving-cert\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394645 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.394618    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-service-ca\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394753 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.394618    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-console-config\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394753 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.394699    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-oauth-serving-cert\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.394950 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.394931    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-trusted-ca-bundle\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.396919 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.396898    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-serving-cert\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.396993 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.396935    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-oauth-config\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.402185 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.402164    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjlz\" (UniqueName: \"kubernetes.io/projected/558988cf-59de-45e5-a7b6-0ad656640e81-kube-api-access-cdjlz\") pod \"console-74844f778b-nq7r9\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") " pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.457125 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.457076    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:44.585906 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:44.585873    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74844f778b-nq7r9"]
Jul 08 02:44:44.588814 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:44:44.588781    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558988cf_59de_45e5_a7b6_0ad656640e81.slice/crio-72365fdcb242b7d91e18947e2ae1ce4ebb3feb193f56677ab08b89bcce2f3af1 WatchSource:0}: Error finding container 72365fdcb242b7d91e18947e2ae1ce4ebb3feb193f56677ab08b89bcce2f3af1: Status 404 returned error can't find the container with id 72365fdcb242b7d91e18947e2ae1ce4ebb3feb193f56677ab08b89bcce2f3af1
Jul 08 02:44:45.507513 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.507473    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74844f778b-nq7r9" event={"ID":"558988cf-59de-45e5-a7b6-0ad656640e81","Type":"ContainerStarted","Data":"72365fdcb242b7d91e18947e2ae1ce4ebb3feb193f56677ab08b89bcce2f3af1"}
Jul 08 02:44:45.849533 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.849448    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rxnl7"]
Jul 08 02:44:45.852818 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.852795    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:45.855356 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.855329    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 02:44:45.855816 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.855788    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f7vjp\""
Jul 08 02:44:45.856397 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.856376    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 02:44:45.856479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.856437    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 02:44:45.856519 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.856437    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 02:44:45.856552 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.856540    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 02:44:45.856657 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:45.856635    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 02:44:46.007526 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17882e9b-f79a-4875-83f3-46a2985ecc79-metrics-client-ca\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-sys\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007630    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-textfile\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007887 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007887 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007775    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-wtmp\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.007887 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-root\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.008014 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.007894    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97xfw\" (UniqueName: \"kubernetes.io/projected/17882e9b-f79a-4875-83f3-46a2985ecc79-kube-api-access-97xfw\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109260 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109176    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17882e9b-f79a-4875-83f3-46a2985ecc79-metrics-client-ca\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109260 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-sys\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109260 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-textfile\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109260 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109250    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109274    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-wtmp\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109327    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-sys\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-root\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109386    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-root\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97xfw\" (UniqueName: \"kubernetes.io/projected/17882e9b-f79a-4875-83f3-46a2985ecc79-kube-api-access-97xfw\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109590 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109532    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-wtmp\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.109974 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:46.109597    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 02:44:46.109974 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:46.109663    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls podName:17882e9b-f79a-4875-83f3-46a2985ecc79 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:46.60964287 +0000 UTC m=+274.537504464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls") pod "node-exporter-rxnl7" (UID: "17882e9b-f79a-4875-83f3-46a2985ecc79") : secret "node-exporter-tls" not found
Jul 08 02:44:46.109974 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109716    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-textfile\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.110125 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.109969    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17882e9b-f79a-4875-83f3-46a2985ecc79-metrics-client-ca\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.110125 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.110055    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.112149 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.112113    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.120218 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.120194    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97xfw\" (UniqueName: \"kubernetes.io/projected/17882e9b-f79a-4875-83f3-46a2985ecc79-kube-api-access-97xfw\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.616346 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:46.616313    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:46.616861 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:46.616469    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 02:44:46.616861 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:46.616539    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls podName:17882e9b-f79a-4875-83f3-46a2985ecc79 nodeName:}" failed. No retries permitted until 2026-07-08 02:44:47.616518001 +0000 UTC m=+275.544379608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls") pod "node-exporter-rxnl7" (UID: "17882e9b-f79a-4875-83f3-46a2985ecc79") : secret "node-exporter-tls" not found
Jul 08 02:44:47.625752 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:47.625704    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:47.628151 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:47.628119    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17882e9b-f79a-4875-83f3-46a2985ecc79-node-exporter-tls\") pod \"node-exporter-rxnl7\" (UID: \"17882e9b-f79a-4875-83f3-46a2985ecc79\") " pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:47.664446 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:47.664407    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rxnl7"
Jul 08 02:44:47.769217 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:44:47.769179    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17882e9b_f79a_4875_83f3_46a2985ecc79.slice/crio-bfceb9272e32f161ad63bc270a926e5d003b121534894a0a596b0d57b0329bdd WatchSource:0}: Error finding container bfceb9272e32f161ad63bc270a926e5d003b121534894a0a596b0d57b0329bdd: Status 404 returned error can't find the container with id bfceb9272e32f161ad63bc270a926e5d003b121534894a0a596b0d57b0329bdd
Jul 08 02:44:48.519598 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:48.519559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxnl7" event={"ID":"17882e9b-f79a-4875-83f3-46a2985ecc79","Type":"ContainerStarted","Data":"bfceb9272e32f161ad63bc270a926e5d003b121534894a0a596b0d57b0329bdd"}
Jul 08 02:44:48.521191 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:48.521157    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74844f778b-nq7r9" event={"ID":"558988cf-59de-45e5-a7b6-0ad656640e81","Type":"ContainerStarted","Data":"76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33"}
Jul 08 02:44:48.538785 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:48.538713    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74844f778b-nq7r9" podStartSLOduration=1.35919766 podStartE2EDuration="4.538694477s" totalImagesPullingTime="3.179496817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:44:48.538161966 +0000 UTC m=+276.466023601" watchObservedRunningTime="2026-07-08 02:44:48.538694477 +0000 UTC m=+276.466556088"
Jul 08 02:44:49.463721 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:49.463670    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" podUID="e300e301-72d3-4dde-b11b-78ea112e2f9d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 08 02:44:49.464091 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:49.463763    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv"
Jul 08 02:44:49.464473 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:49.464432    2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"2bef56525a49cf39c051b6025e94ab65f549e44843749e1abd8bd329a1bc86ac"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 08 02:44:49.464531 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:49.464503    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" podUID="e300e301-72d3-4dde-b11b-78ea112e2f9d" containerName="service-proxy" containerID="cri-o://2bef56525a49cf39c051b6025e94ab65f549e44843749e1abd8bd329a1bc86ac" gracePeriod=30
Jul 08 02:44:49.526401 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:49.526360    2558 generic.go:356] "Generic (PLEG): container finished" podID="17882e9b-f79a-4875-83f3-46a2985ecc79" containerID="68ec577486a3102f7dc02b4092d79f312d5dbb9baf49a1202a56606d830b5e62" exitCode=0
Jul 08 02:44:49.526523 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:49.526418    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxnl7" event={"ID":"17882e9b-f79a-4875-83f3-46a2985ecc79","Type":"ContainerDied","Data":"68ec577486a3102f7dc02b4092d79f312d5dbb9baf49a1202a56606d830b5e62"}
Jul 08 02:44:50.531934 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:50.531897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxnl7" event={"ID":"17882e9b-f79a-4875-83f3-46a2985ecc79","Type":"ContainerStarted","Data":"4ffff3cf63150a0bab1c6c25e52b398e0f69a45af3df21363063a7aff8c328a1"}
Jul 08 02:44:50.532468 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:50.531953    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxnl7" event={"ID":"17882e9b-f79a-4875-83f3-46a2985ecc79","Type":"ContainerStarted","Data":"26fd74fc315eb78df3f13d87d6756b9b7064631d6d19baa45982ef644ed39135"}
Jul 08 02:44:50.534315 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:50.534287    2558 generic.go:356] "Generic (PLEG): container finished" podID="e300e301-72d3-4dde-b11b-78ea112e2f9d" containerID="2bef56525a49cf39c051b6025e94ab65f549e44843749e1abd8bd329a1bc86ac" exitCode=2
Jul 08 02:44:50.534419 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:50.534322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" event={"ID":"e300e301-72d3-4dde-b11b-78ea112e2f9d","Type":"ContainerDied","Data":"2bef56525a49cf39c051b6025e94ab65f549e44843749e1abd8bd329a1bc86ac"}
Jul 08 02:44:50.534419 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:50.534345    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-675944bb65-qjqjv" event={"ID":"e300e301-72d3-4dde-b11b-78ea112e2f9d","Type":"ContainerStarted","Data":"d866e0f092bb32690528fa61ba7942e685165a99a1e4121b1cc4915ae1c4e264"}
Jul 08 02:44:50.552595 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:50.552549    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rxnl7" podStartSLOduration=3.990331039 podStartE2EDuration="5.552533771s" totalImagesPullingTime="1.472546679s" totalInitContainerRuntime="89.656053ms" isStatefulPod=true podCreationTimestamp="2026-07-08 02:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:44:50.55154095 +0000 UTC m=+278.479402561" watchObservedRunningTime="2026-07-08 02:44:50.552533771 +0000 UTC m=+278.480395380"
Jul 08 02:44:51.103976 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:51.103926    2558 patch_prober.go:28] interesting pod/image-registry-5b74d44cc5-n5rrr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 02:44:51.104140 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:51.104000    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 02:44:52.469305 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:52.469271    2558 patch_prober.go:28] interesting pod/image-registry-5b74d44cc5-n5rrr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 02:44:52.469687 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:52.469323    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 02:44:54.156103 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:44:54.156055    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f8hwx" podUID="d011ddb5-0557-4f54-b4a5-22abc3e9798c"
Jul 08 02:44:54.457525 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:54.457487    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:54.457832 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:54.457804    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:44:54.459023 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:54.458999    2558 patch_prober.go:28] interesting pod/console-74844f778b-nq7r9 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.12:8443/health\": dial tcp 10.134.0.12:8443: connect: connection refused" start-of-body=
Jul 08 02:44:54.459130 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:54.459042    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-74844f778b-nq7r9" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" containerName="console" probeResult="failure" output="Get \"https://10.134.0.12:8443/health\": dial tcp 10.134.0.12:8443: connect: connection refused"
Jul 08 02:44:54.547968 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:54.547934    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:44:57.399922 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:57.399885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:44:57.402337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:57.402311    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d011ddb5-0557-4f54-b4a5-22abc3e9798c-metrics-tls\") pod \"dns-default-f8hwx\" (UID: \"d011ddb5-0557-4f54-b4a5-22abc3e9798c\") " pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:44:57.551904 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:57.551874    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8d2rg\""
Jul 08 02:44:57.559740 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:57.559720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:44:57.684490 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:57.684456    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f8hwx"]
Jul 08 02:44:57.688645 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:44:57.688616    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd011ddb5_0557_4f54_b4a5_22abc3e9798c.slice/crio-b2a5c63d02a6c4d7591fed8358a3f92cec5cdcd5701edc046fc284d2758b1222 WatchSource:0}: Error finding container b2a5c63d02a6c4d7591fed8358a3f92cec5cdcd5701edc046fc284d2758b1222: Status 404 returned error can't find the container with id b2a5c63d02a6c4d7591fed8358a3f92cec5cdcd5701edc046fc284d2758b1222
Jul 08 02:44:58.562200 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:58.562154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8hwx" event={"ID":"d011ddb5-0557-4f54-b4a5-22abc3e9798c","Type":"ContainerStarted","Data":"b2a5c63d02a6c4d7591fed8358a3f92cec5cdcd5701edc046fc284d2758b1222"}
Jul 08 02:44:59.567561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:59.567468    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8hwx" event={"ID":"d011ddb5-0557-4f54-b4a5-22abc3e9798c","Type":"ContainerStarted","Data":"f6d3cdb5628d5c2713520288b47aa08b43e8c53a0e9b680343f0a96db8725af0"}
Jul 08 02:44:59.567561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:59.567527    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8hwx" event={"ID":"d011ddb5-0557-4f54-b4a5-22abc3e9798c","Type":"ContainerStarted","Data":"9b05b8475f1e94e00aa6ccfaffff5f0ca9051a52dd857811f3bcb173d45b812c"}
Jul 08 02:44:59.568076 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:59.567615    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:44:59.584935 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:44:59.584887    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f8hwx" podStartSLOduration=251.024449548 podStartE2EDuration="4m12.584871855s" totalImagesPullingTime="1.560422307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:44:59.583094651 +0000 UTC m=+287.510956274" watchObservedRunningTime="2026-07-08 02:44:59.584871855 +0000 UTC m=+287.512733465"
Jul 08 02:45:01.103771 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:01.103734    2558 patch_prober.go:28] interesting pod/image-registry-5b74d44cc5-n5rrr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 02:45:01.104218 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:01.103790    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 02:45:02.468746 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:02.468708    2558 patch_prober.go:28] interesting pod/image-registry-5b74d44cc5-n5rrr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 02:45:02.469130 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:02.468762    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 02:45:04.457783 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:04.457742    2558 patch_prober.go:28] interesting pod/console-74844f778b-nq7r9 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.12:8443/health\": dial tcp 10.134.0.12:8443: connect: connection refused" start-of-body=
Jul 08 02:45:04.458190 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:04.457803    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-74844f778b-nq7r9" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" containerName="console" probeResult="failure" output="Get \"https://10.134.0.12:8443/health\": dial tcp 10.134.0.12:8443: connect: connection refused"
Jul 08 02:45:09.573093 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:09.573065    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f8hwx"
Jul 08 02:45:11.050860 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.050811    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74844f778b-nq7r9"]
Jul 08 02:45:11.104031 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.103998    2558 patch_prober.go:28] interesting pod/image-registry-5b74d44cc5-n5rrr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 02:45:11.104190 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.104056    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 02:45:11.104190 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.104092    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:45:11.104625 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.104602    2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"5effdd724467f955232a0588a66b51c9b428d186fd6beb0cfd3f26f2ad0978c7"} pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 08 02:45:11.108141 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.108101    2558 patch_prober.go:28] interesting pod/image-registry-5b74d44cc5-n5rrr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 02:45:11.108270 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:11.108157    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 02:45:12.457927 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:12.457896    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:45:12.459373 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:12.459352    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:45:12.460758 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:12.460739    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 02:45:21.108582 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:21.108553    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:45:36.072404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.072359    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74844f778b-nq7r9" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" containerName="console" containerID="cri-o://76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33" gracePeriod=15
Jul 08 02:45:36.124252 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.124212    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" podUID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerName="registry" containerID="cri-o://5effdd724467f955232a0588a66b51c9b428d186fd6beb0cfd3f26f2ad0978c7" gracePeriod=30
Jul 08 02:45:36.308247 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.308224    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:45:36.382924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.382829    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-serving-cert\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.382924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.382882    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-service-ca\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.382924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.382921    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdjlz\" (UniqueName: \"kubernetes.io/projected/558988cf-59de-45e5-a7b6-0ad656640e81-kube-api-access-cdjlz\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.383184 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.382954    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-console-config\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.383184 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.382969    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-trusted-ca-bundle\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.383184 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.382984    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-oauth-serving-cert\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.383184 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.383018    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-oauth-config\") pod \"558988cf-59de-45e5-a7b6-0ad656640e81\" (UID: \"558988cf-59de-45e5-a7b6-0ad656640e81\") "
Jul 08 02:45:36.383465 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.383326    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-service-ca" (OuterVolumeSpecName: "service-ca") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:45:36.383465 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.383396    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-console-config" (OuterVolumeSpecName: "console-config") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:45:36.383465 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.383409    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:45:36.383465 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.383450    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:45:36.385588 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.385552    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 02:45:36.385588 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.385570    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 02:45:36.385588 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.385559    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558988cf-59de-45e5-a7b6-0ad656640e81-kube-api-access-cdjlz" (OuterVolumeSpecName: "kube-api-access-cdjlz") pod "558988cf-59de-45e5-a7b6-0ad656640e81" (UID: "558988cf-59de-45e5-a7b6-0ad656640e81"). InnerVolumeSpecName "kube-api-access-cdjlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:45:36.484258 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484219    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-console-config\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.484258 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484249    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-trusted-ca-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.484258 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484262    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-oauth-serving-cert\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.484258 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484270    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-oauth-config\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.484511 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484279    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558988cf-59de-45e5-a7b6-0ad656640e81-console-serving-cert\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.484511 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484287    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558988cf-59de-45e5-a7b6-0ad656640e81-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.484511 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.484295    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjlz\" (UniqueName: \"kubernetes.io/projected/558988cf-59de-45e5-a7b6-0ad656640e81-kube-api-access-cdjlz\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:45:36.694715 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.694674    2558 generic.go:356] "Generic (PLEG): container finished" podID="558988cf-59de-45e5-a7b6-0ad656640e81" containerID="76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33" exitCode=0
Jul 08 02:45:36.694906 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.694745    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74844f778b-nq7r9"
Jul 08 02:45:36.694906 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.694754    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74844f778b-nq7r9" event={"ID":"558988cf-59de-45e5-a7b6-0ad656640e81","Type":"ContainerDied","Data":"76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33"}
Jul 08 02:45:36.694906 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.694822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74844f778b-nq7r9" event={"ID":"558988cf-59de-45e5-a7b6-0ad656640e81","Type":"ContainerDied","Data":"72365fdcb242b7d91e18947e2ae1ce4ebb3feb193f56677ab08b89bcce2f3af1"}
Jul 08 02:45:36.694906 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.694867    2558 scope.go:117] "RemoveContainer" containerID="76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33"
Jul 08 02:45:36.703091 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.703071    2558 scope.go:117] "RemoveContainer" containerID="76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33"
Jul 08 02:45:36.703399 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:45:36.703376    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33\": container with ID starting with 76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33 not found: ID does not exist" containerID="76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33"
Jul 08 02:45:36.703445 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.703407    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33"} err="failed to get container status \"76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33\": rpc error: code = NotFound desc = could not find container \"76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33\": container with ID starting with 76ea83c4027e25d4d968c5f82276e43bb355673a574ec276569f76b4bc886a33 not found: ID does not exist"
Jul 08 02:45:36.711098 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.711069    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74844f778b-nq7r9"]
Jul 08 02:45:36.713777 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:36.713753    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74844f778b-nq7r9"]
Jul 08 02:45:37.701018 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:37.700986    2558 generic.go:356] "Generic (PLEG): container finished" podID="14b3f5ab-1cb9-45c6-a468-8f539636528e" containerID="5effdd724467f955232a0588a66b51c9b428d186fd6beb0cfd3f26f2ad0978c7" exitCode=0
Jul 08 02:45:37.701018 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:37.701022    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" event={"ID":"14b3f5ab-1cb9-45c6-a468-8f539636528e","Type":"ContainerDied","Data":"5effdd724467f955232a0588a66b51c9b428d186fd6beb0cfd3f26f2ad0978c7"}
Jul 08 02:45:37.701422 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:37.701055    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr" event={"ID":"14b3f5ab-1cb9-45c6-a468-8f539636528e","Type":"ContainerStarted","Data":"f1fd0316cb5b76e80375f1f7a1e455062d13a9dc9e397b1f56033c3d159ca1a7"}
Jul 08 02:45:37.701422 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:37.701094    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:45:38.617669 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:38.617634    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" path="/var/lib/kubelet/pods/558988cf-59de-45e5-a7b6-0ad656640e81/volumes"
Jul 08 02:45:58.708973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:45:58.708943    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b74d44cc5-n5rrr"
Jul 08 02:46:14.373690 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.373655    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bb47f8679-7l2jl"]
Jul 08 02:46:14.374146 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.373918    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" containerName="console"
Jul 08 02:46:14.374146 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.373931    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" containerName="console"
Jul 08 02:46:14.374146 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.373977    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="558988cf-59de-45e5-a7b6-0ad656640e81" containerName="console"
Jul 08 02:46:14.377973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.377956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.386019 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.385981    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 02:46:14.386136 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.386065    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 02:46:14.386980 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.386964    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 02:46:14.387086 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.386984    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ws27b\""
Jul 08 02:46:14.387086 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.386985    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 02:46:14.387961 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.387946    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 02:46:14.388032 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.387953    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 02:46:14.389734 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.389719    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 02:46:14.421173 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.421146    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bb47f8679-7l2jl"]
Jul 08 02:46:14.429033 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.429011    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 02:46:14.551942 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.551892    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-serving-cert\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.552135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.551964    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-console-config\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.552135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.551982    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-oauth-serving-cert\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.552135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.552016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-oauth-config\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.552135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.552030    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-service-ca\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.552135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.552070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-trusted-ca-bundle\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.552135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.552102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhc4q\" (UniqueName: \"kubernetes.io/projected/fad03b10-8923-4bb3-856a-a70c6a19f284-kube-api-access-zhc4q\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-serving-cert\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653399    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-console-config\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653416    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-oauth-serving-cert\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653677 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-oauth-config\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653677 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653472    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-service-ca\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653677 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-trusted-ca-bundle\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.653677 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.653520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhc4q\" (UniqueName: \"kubernetes.io/projected/fad03b10-8923-4bb3-856a-a70c6a19f284-kube-api-access-zhc4q\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.654245 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.654217    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-oauth-serving-cert\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.654327 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.654243    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-console-config\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.654532 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.654512    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-service-ca\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.654818 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.654795    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-trusted-ca-bundle\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.656114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.656093    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-serving-cert\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.656223 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.656174    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-oauth-config\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.661221 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.661197    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhc4q\" (UniqueName: \"kubernetes.io/projected/fad03b10-8923-4bb3-856a-a70c6a19f284-kube-api-access-zhc4q\") pod \"console-5bb47f8679-7l2jl\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") " pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.687160 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.687121    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:14.817919 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:14.817871    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bb47f8679-7l2jl"]
Jul 08 02:46:14.822458 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:46:14.822431    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad03b10_8923_4bb3_856a_a70c6a19f284.slice/crio-9ed95a2c1f75f42a07bc1970acba3c9ffda4f022dc06445db15e8031c769f49f WatchSource:0}: Error finding container 9ed95a2c1f75f42a07bc1970acba3c9ffda4f022dc06445db15e8031c769f49f: Status 404 returned error can't find the container with id 9ed95a2c1f75f42a07bc1970acba3c9ffda4f022dc06445db15e8031c769f49f
Jul 08 02:46:15.824111 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:15.824070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb47f8679-7l2jl" event={"ID":"fad03b10-8923-4bb3-856a-a70c6a19f284","Type":"ContainerStarted","Data":"7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071"}
Jul 08 02:46:15.824522 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:15.824156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb47f8679-7l2jl" event={"ID":"fad03b10-8923-4bb3-856a-a70c6a19f284","Type":"ContainerStarted","Data":"9ed95a2c1f75f42a07bc1970acba3c9ffda4f022dc06445db15e8031c769f49f"}
Jul 08 02:46:15.849321 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:15.849263    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bb47f8679-7l2jl" podStartSLOduration=1.8492468020000001 podStartE2EDuration="1.849246802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:46:15.848591774 +0000 UTC m=+363.776453383" watchObservedRunningTime="2026-07-08 02:46:15.849246802 +0000 UTC m=+363.777108411"
Jul 08 02:46:24.687548 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:24.687503    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:24.688180 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:24.687581    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:24.692327 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:24.692306    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:24.858202 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:24.858172    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:46:35.623954 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:46:35.623918    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bb47f8679-7l2jl"]
Jul 08 02:47:00.644267 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.644224    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5bb47f8679-7l2jl" podUID="fad03b10-8923-4bb3-856a-a70c6a19f284" containerName="console" containerID="cri-o://7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071" gracePeriod=15
Jul 08 02:47:00.883361 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.883337    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:47:00.975159 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.975118    2558 generic.go:356] "Generic (PLEG): container finished" podID="fad03b10-8923-4bb3-856a-a70c6a19f284" containerID="7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071" exitCode=0
Jul 08 02:47:00.975159 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.975161    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb47f8679-7l2jl" event={"ID":"fad03b10-8923-4bb3-856a-a70c6a19f284","Type":"ContainerDied","Data":"7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071"}
Jul 08 02:47:00.975411 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.975170    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb47f8679-7l2jl"
Jul 08 02:47:00.975411 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.975211    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb47f8679-7l2jl" event={"ID":"fad03b10-8923-4bb3-856a-a70c6a19f284","Type":"ContainerDied","Data":"9ed95a2c1f75f42a07bc1970acba3c9ffda4f022dc06445db15e8031c769f49f"}
Jul 08 02:47:00.975411 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.975235    2558 scope.go:117] "RemoveContainer" containerID="7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071"
Jul 08 02:47:00.982902 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.982882    2558 scope.go:117] "RemoveContainer" containerID="7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071"
Jul 08 02:47:00.983172 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:47:00.983153    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071\": container with ID starting with 7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071 not found: ID does not exist" containerID="7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071"
Jul 08 02:47:00.983215 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.983182    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071"} err="failed to get container status \"7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071\": rpc error: code = NotFound desc = could not find container \"7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071\": container with ID starting with 7016d0649db1fa1ab96c05aeac16adac90e968378afa97cddad7e8fb293b1071 not found: ID does not exist"
Jul 08 02:47:00.988423 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988406    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-serving-cert\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988486 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988444    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-oauth-serving-cert\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988486 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988467    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhc4q\" (UniqueName: \"kubernetes.io/projected/fad03b10-8923-4bb3-856a-a70c6a19f284-kube-api-access-zhc4q\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988583 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988499    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-oauth-config\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988637 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988609    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-service-ca\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988687 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988644    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-trusted-ca-bundle\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988737 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988692    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-console-config\") pod \"fad03b10-8923-4bb3-856a-a70c6a19f284\" (UID: \"fad03b10-8923-4bb3-856a-a70c6a19f284\") "
Jul 08 02:47:00.988968 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.988808    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:47:00.989100 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.989016    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-service-ca" (OuterVolumeSpecName: "service-ca") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:47:00.989179 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.989156    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:47:00.989357 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.989338    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-console-config" (OuterVolumeSpecName: "console-config") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 02:47:00.990645 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.990626    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 02:47:00.991152 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.991132    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad03b10-8923-4bb3-856a-a70c6a19f284-kube-api-access-zhc4q" (OuterVolumeSpecName: "kube-api-access-zhc4q") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "kube-api-access-zhc4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:47:00.991263 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:00.991144    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fad03b10-8923-4bb3-856a-a70c6a19f284" (UID: "fad03b10-8923-4bb3-856a-a70c6a19f284"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 02:47:01.089661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089624    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-serving-cert\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.089661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089653    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-oauth-serving-cert\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.089661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089663    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhc4q\" (UniqueName: \"kubernetes.io/projected/fad03b10-8923-4bb3-856a-a70c6a19f284-kube-api-access-zhc4q\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.089661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089672    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fad03b10-8923-4bb3-856a-a70c6a19f284-console-oauth-config\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.089945 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089697    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.089945 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089708    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-trusted-ca-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.089945 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.089716    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fad03b10-8923-4bb3-856a-a70c6a19f284-console-config\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:47:01.305249 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.305219    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bb47f8679-7l2jl"]
Jul 08 02:47:01.309510 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:01.309487    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bb47f8679-7l2jl"]
Jul 08 02:47:02.618496 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:02.618463    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad03b10-8923-4bb3-856a-a70c6a19f284" path="/var/lib/kubelet/pods/fad03b10-8923-4bb3-856a-a70c6a19f284/volumes"
Jul 08 02:47:40.636022 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.635947    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"]
Jul 08 02:47:40.636523 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.636202    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad03b10-8923-4bb3-856a-a70c6a19f284" containerName="console"
Jul 08 02:47:40.636523 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.636217    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad03b10-8923-4bb3-856a-a70c6a19f284" containerName="console"
Jul 08 02:47:40.636523 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.636271    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad03b10-8923-4bb3-856a-a70c6a19f284" containerName="console"
Jul 08 02:47:40.639004 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.638988    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.642105 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.642083    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-fb5gk\""
Jul 08 02:47:40.642794 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.642778    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 08 02:47:40.643146 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.643126    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 08 02:47:40.659798 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.659773    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"]
Jul 08 02:47:40.769683 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.769638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a982546-b37f-4c7f-b949-883b5d567079-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnr5n\" (UID: \"6a982546-b37f-4c7f-b949-883b5d567079\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.769905 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.769739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmts9\" (UniqueName: \"kubernetes.io/projected/6a982546-b37f-4c7f-b949-883b5d567079-kube-api-access-qmts9\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnr5n\" (UID: \"6a982546-b37f-4c7f-b949-883b5d567079\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.871142 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.871102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmts9\" (UniqueName: \"kubernetes.io/projected/6a982546-b37f-4c7f-b949-883b5d567079-kube-api-access-qmts9\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnr5n\" (UID: \"6a982546-b37f-4c7f-b949-883b5d567079\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.871343 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.871171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a982546-b37f-4c7f-b949-883b5d567079-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnr5n\" (UID: \"6a982546-b37f-4c7f-b949-883b5d567079\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.871507 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.871490    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a982546-b37f-4c7f-b949-883b5d567079-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnr5n\" (UID: \"6a982546-b37f-4c7f-b949-883b5d567079\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.884116 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.884087    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmts9\" (UniqueName: \"kubernetes.io/projected/6a982546-b37f-4c7f-b949-883b5d567079-kube-api-access-qmts9\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnr5n\" (UID: \"6a982546-b37f-4c7f-b949-883b5d567079\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:40.947911 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:40.947880    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"
Jul 08 02:47:41.101205 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:41.100759    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n"]
Jul 08 02:47:41.104951 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:47:41.104826    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a982546_b37f_4c7f_b949_883b5d567079.slice/crio-13fada11b3fd133958f2294c7140f9d90baf57abee4d71e34abc9ef09c5797b9 WatchSource:0}: Error finding container 13fada11b3fd133958f2294c7140f9d90baf57abee4d71e34abc9ef09c5797b9: Status 404 returned error can't find the container with id 13fada11b3fd133958f2294c7140f9d90baf57abee4d71e34abc9ef09c5797b9
Jul 08 02:47:41.106720 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:41.106697    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 02:47:42.114673 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:42.114633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n" event={"ID":"6a982546-b37f-4c7f-b949-883b5d567079","Type":"ContainerStarted","Data":"13fada11b3fd133958f2294c7140f9d90baf57abee4d71e34abc9ef09c5797b9"}
Jul 08 02:47:45.127460 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.127408    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n" event={"ID":"6a982546-b37f-4c7f-b949-883b5d567079","Type":"ContainerStarted","Data":"56ef00ac165552be65417f208b129dcd18f541cd7dc23e1bf5b8f5e77ea6edc9"}
Jul 08 02:47:45.162014 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.161965    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnr5n" podStartSLOduration=2.0292954 podStartE2EDuration="5.161948695s" totalImagesPullingTime="3.132653295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:47:45.161685322 +0000 UTC m=+453.089546931" watchObservedRunningTime="2026-07-08 02:47:45.161948695 +0000 UTC m=+453.089810305"
Jul 08 02:47:45.912525 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.912489    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"]
Jul 08 02:47:45.915769 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.915746    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:45.920813 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.920785    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 08 02:47:45.920981 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.920876    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v8mtf\""
Jul 08 02:47:45.921796 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.921782    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 08 02:47:45.935564 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:45.935530    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"]
Jul 08 02:47:46.009377 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.009339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pq29\" (UniqueName: \"kubernetes.io/projected/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-kube-api-access-4pq29\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.009561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.009382    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.009561 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.009455    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.110858 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.110811    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pq29\" (UniqueName: \"kubernetes.io/projected/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-kube-api-access-4pq29\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.110858 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.110865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.111117 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.110892    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.111249 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.111233    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.111303 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.111266    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.134557 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.134526    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pq29\" (UniqueName: \"kubernetes.io/projected/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-kube-api-access-4pq29\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.225259 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.225225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:47:46.418201 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:46.418161    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"]
Jul 08 02:47:46.421950 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:47:46.421916    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaabad0f8_b144_4130_a20d_7ff1a9e3eee5.slice/crio-c41c92b18219fd1835f724fd884347846208c5b9cc2697bc3ba8365c22a63a7a WatchSource:0}: Error finding container c41c92b18219fd1835f724fd884347846208c5b9cc2697bc3ba8365c22a63a7a: Status 404 returned error can't find the container with id c41c92b18219fd1835f724fd884347846208c5b9cc2697bc3ba8365c22a63a7a
Jul 08 02:47:47.136706 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:47.136663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5" event={"ID":"aabad0f8-b144-4130-a20d-7ff1a9e3eee5","Type":"ContainerStarted","Data":"c41c92b18219fd1835f724fd884347846208c5b9cc2697bc3ba8365c22a63a7a"}
Jul 08 02:47:53.160273 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:53.160229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5" event={"ID":"aabad0f8-b144-4130-a20d-7ff1a9e3eee5","Type":"ContainerStarted","Data":"5d7582122512f7388c07113cb465ea62499b94f16e213c80ac632a7e17f2c5b9"}
Jul 08 02:47:54.165032 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:54.164994    2558 generic.go:356] "Generic (PLEG): container finished" podID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerID="5d7582122512f7388c07113cb465ea62499b94f16e213c80ac632a7e17f2c5b9" exitCode=0
Jul 08 02:47:54.165409 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:54.165061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5" event={"ID":"aabad0f8-b144-4130-a20d-7ff1a9e3eee5","Type":"ContainerDied","Data":"5d7582122512f7388c07113cb465ea62499b94f16e213c80ac632a7e17f2c5b9"}
Jul 08 02:47:57.178460 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:57.178415    2558 generic.go:356] "Generic (PLEG): container finished" podID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerID="c1cf6a5c04d09370578ebfa47fcd7d8d0676044961d95d09bf2146682d7afda5" exitCode=0
Jul 08 02:47:57.178833 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:47:57.178502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5" event={"ID":"aabad0f8-b144-4130-a20d-7ff1a9e3eee5","Type":"ContainerDied","Data":"c1cf6a5c04d09370578ebfa47fcd7d8d0676044961d95d09bf2146682d7afda5"}
Jul 08 02:48:04.210574 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:04.210537    2558 generic.go:356] "Generic (PLEG): container finished" podID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerID="cc4233e0ac4bdb7e733e85961f720c95f2b5e0734ebcd134179039b05f11b042" exitCode=0
Jul 08 02:48:04.210962 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:04.210601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5" event={"ID":"aabad0f8-b144-4130-a20d-7ff1a9e3eee5","Type":"ContainerDied","Data":"cc4233e0ac4bdb7e733e85961f720c95f2b5e0734ebcd134179039b05f11b042"}
Jul 08 02:48:05.338435 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.338400    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:48:05.358594 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.358562    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-bundle\") pod \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") "
Jul 08 02:48:05.358750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.358635    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pq29\" (UniqueName: \"kubernetes.io/projected/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-kube-api-access-4pq29\") pod \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") "
Jul 08 02:48:05.358750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.358680    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-util\") pod \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\" (UID: \"aabad0f8-b144-4130-a20d-7ff1a9e3eee5\") "
Jul 08 02:48:05.359055 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.359025    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-bundle" (OuterVolumeSpecName: "bundle") pod "aabad0f8-b144-4130-a20d-7ff1a9e3eee5" (UID: "aabad0f8-b144-4130-a20d-7ff1a9e3eee5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:48:05.361448 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.361416    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-kube-api-access-4pq29" (OuterVolumeSpecName: "kube-api-access-4pq29") pod "aabad0f8-b144-4130-a20d-7ff1a9e3eee5" (UID: "aabad0f8-b144-4130-a20d-7ff1a9e3eee5"). InnerVolumeSpecName "kube-api-access-4pq29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:48:05.363753 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.363728    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-util" (OuterVolumeSpecName: "util") pod "aabad0f8-b144-4130-a20d-7ff1a9e3eee5" (UID: "aabad0f8-b144-4130-a20d-7ff1a9e3eee5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:48:05.459182 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.459148    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:48:05.459182 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.459179    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pq29\" (UniqueName: \"kubernetes.io/projected/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-kube-api-access-4pq29\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:48:05.459182 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:05.459191    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aabad0f8-b144-4130-a20d-7ff1a9e3eee5-util\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:48:06.221348 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:06.221315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5" event={"ID":"aabad0f8-b144-4130-a20d-7ff1a9e3eee5","Type":"ContainerDied","Data":"c41c92b18219fd1835f724fd884347846208c5b9cc2697bc3ba8365c22a63a7a"}
Jul 08 02:48:06.221512 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:06.221359    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsznt5"
Jul 08 02:48:06.221512 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:06.221368    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c41c92b18219fd1835f724fd884347846208c5b9cc2697bc3ba8365c22a63a7a"
Jul 08 02:48:18.731629 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.731584    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"]
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.731948    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="util"
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.731966    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="util"
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.731980    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="pull"
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.731989    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="pull"
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.731999    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="extract"
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.732008    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="extract"
Jul 08 02:48:18.732233 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.732086    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="aabad0f8-b144-4130-a20d-7ff1a9e3eee5" containerName="extract"
Jul 08 02:48:18.737050 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.737023    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.739652 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.739628    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 08 02:48:18.739778 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.739628    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 08 02:48:18.740672 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.740656    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v8mtf\""
Jul 08 02:48:18.743236 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.743211    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"]
Jul 08 02:48:18.859112 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.859073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xht8\" (UniqueName: \"kubernetes.io/projected/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-kube-api-access-4xht8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.859311 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.859121    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.859311 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.859152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.959864 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.959808    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.960056 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.959882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.960056 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.959951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xht8\" (UniqueName: \"kubernetes.io/projected/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-kube-api-access-4xht8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.960240 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.960218    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.960306 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.960288    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:18.968833 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:18.968799    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xht8\" (UniqueName: \"kubernetes.io/projected/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-kube-api-access-4xht8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:19.051896 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:19.051781    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:19.179154 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:19.179121    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"]
Jul 08 02:48:19.182333 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:48:19.182307    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a7c7cf0_35b3_46fa_98b1_c89c6974b995.slice/crio-3b67f837e4633c31c9a2b07108ea37d50789b7413cbf1bdc2e12ec10bcd8b36a WatchSource:0}: Error finding container 3b67f837e4633c31c9a2b07108ea37d50789b7413cbf1bdc2e12ec10bcd8b36a: Status 404 returned error can't find the container with id 3b67f837e4633c31c9a2b07108ea37d50789b7413cbf1bdc2e12ec10bcd8b36a
Jul 08 02:48:19.267930 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:19.267893    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2" event={"ID":"6a7c7cf0-35b3-46fa-98b1-c89c6974b995","Type":"ContainerStarted","Data":"c2f8d94f57cabe452645ded332f007fdc6025b30a554023df69acf7082b4e382"}
Jul 08 02:48:19.268100 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:19.267955    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2" event={"ID":"6a7c7cf0-35b3-46fa-98b1-c89c6974b995","Type":"ContainerStarted","Data":"3b67f837e4633c31c9a2b07108ea37d50789b7413cbf1bdc2e12ec10bcd8b36a"}
Jul 08 02:48:20.273735 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:20.273657    2558 generic.go:356] "Generic (PLEG): container finished" podID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerID="c2f8d94f57cabe452645ded332f007fdc6025b30a554023df69acf7082b4e382" exitCode=0
Jul 08 02:48:20.274150 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:20.273750    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2" event={"ID":"6a7c7cf0-35b3-46fa-98b1-c89c6974b995","Type":"ContainerDied","Data":"c2f8d94f57cabe452645ded332f007fdc6025b30a554023df69acf7082b4e382"}
Jul 08 02:48:21.279492 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:21.279457    2558 generic.go:356] "Generic (PLEG): container finished" podID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerID="dda8353efebaaa4fd54bbb2f0e1db00e91de940f9c5c1eb1c41ef67608d86971" exitCode=0
Jul 08 02:48:21.279894 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:21.279547    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2" event={"ID":"6a7c7cf0-35b3-46fa-98b1-c89c6974b995","Type":"ContainerDied","Data":"dda8353efebaaa4fd54bbb2f0e1db00e91de940f9c5c1eb1c41ef67608d86971"}
Jul 08 02:48:22.285859 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:22.285806    2558 generic.go:356] "Generic (PLEG): container finished" podID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerID="c7b6b4e1dc4974fdbb4fd06d897a92daf52271e39cec78926fdf59ee4a0b35ee" exitCode=0
Jul 08 02:48:22.286226 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:22.285892    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2" event={"ID":"6a7c7cf0-35b3-46fa-98b1-c89c6974b995","Type":"ContainerDied","Data":"c7b6b4e1dc4974fdbb4fd06d897a92daf52271e39cec78926fdf59ee4a0b35ee"}
Jul 08 02:48:23.413937 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.413910    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:23.493292 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.493249    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-bundle\") pod \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") "
Jul 08 02:48:23.493485 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.493385    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xht8\" (UniqueName: \"kubernetes.io/projected/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-kube-api-access-4xht8\") pod \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") "
Jul 08 02:48:23.493485 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.493421    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-util\") pod \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\" (UID: \"6a7c7cf0-35b3-46fa-98b1-c89c6974b995\") "
Jul 08 02:48:23.494114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.494067    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-bundle" (OuterVolumeSpecName: "bundle") pod "6a7c7cf0-35b3-46fa-98b1-c89c6974b995" (UID: "6a7c7cf0-35b3-46fa-98b1-c89c6974b995"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:48:23.495573 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.495551    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-kube-api-access-4xht8" (OuterVolumeSpecName: "kube-api-access-4xht8") pod "6a7c7cf0-35b3-46fa-98b1-c89c6974b995" (UID: "6a7c7cf0-35b3-46fa-98b1-c89c6974b995"). InnerVolumeSpecName "kube-api-access-4xht8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:48:23.499519 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.499493    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-util" (OuterVolumeSpecName: "util") pod "6a7c7cf0-35b3-46fa-98b1-c89c6974b995" (UID: "6a7c7cf0-35b3-46fa-98b1-c89c6974b995"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:48:23.594186 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.594092    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:48:23.594186 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.594120    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xht8\" (UniqueName: \"kubernetes.io/projected/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-kube-api-access-4xht8\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:48:23.594186 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:23.594132    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7c7cf0-35b3-46fa-98b1-c89c6974b995-util\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:48:24.295246 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:24.295204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2" event={"ID":"6a7c7cf0-35b3-46fa-98b1-c89c6974b995","Type":"ContainerDied","Data":"3b67f837e4633c31c9a2b07108ea37d50789b7413cbf1bdc2e12ec10bcd8b36a"}
Jul 08 02:48:24.295246 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:24.295240    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmdrh2"
Jul 08 02:48:24.295246 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:24.295254    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b67f837e4633c31c9a2b07108ea37d50789b7413cbf1bdc2e12ec10bcd8b36a"
Jul 08 02:48:31.891109 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891071    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"]
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891318    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="extract"
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891329    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="extract"
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891338    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="util"
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891344    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="util"
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891364    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="pull"
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891370    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="pull"
Jul 08 02:48:31.891479 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.891429    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a7c7cf0-35b3-46fa-98b1-c89c6974b995" containerName="extract"
Jul 08 02:48:31.893496 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.893472    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:31.896592 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.896556    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 08 02:48:31.896801 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.896785    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 08 02:48:31.897547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.897529    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 08 02:48:31.897882 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.897866    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7497w\""
Jul 08 02:48:31.897970 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.897898    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 08 02:48:31.897970 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.897903    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 08 02:48:31.910269 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.910199    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"]
Jul 08 02:48:31.954632 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.954599    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b189f01-3984-4478-a7cf-9aa1e0d36a43-metrics-cert\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:31.954806 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.954645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b189f01-3984-4478-a7cf-9aa1e0d36a43-cert\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:31.954806 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.954665    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b189f01-3984-4478-a7cf-9aa1e0d36a43-manager-config\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:31.954806 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:31.954735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhz6n\" (UniqueName: \"kubernetes.io/projected/9b189f01-3984-4478-a7cf-9aa1e0d36a43-kube-api-access-dhz6n\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.055973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.055935    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b189f01-3984-4478-a7cf-9aa1e0d36a43-metrics-cert\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.055973 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.055982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b189f01-3984-4478-a7cf-9aa1e0d36a43-cert\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.056223 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.056001    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b189f01-3984-4478-a7cf-9aa1e0d36a43-manager-config\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.056223 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.056065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhz6n\" (UniqueName: \"kubernetes.io/projected/9b189f01-3984-4478-a7cf-9aa1e0d36a43-kube-api-access-dhz6n\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.056739 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.056702    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b189f01-3984-4478-a7cf-9aa1e0d36a43-manager-config\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.058563 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.058544    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b189f01-3984-4478-a7cf-9aa1e0d36a43-cert\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.058622 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.058594    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b189f01-3984-4478-a7cf-9aa1e0d36a43-metrics-cert\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.068430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.068409    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhz6n\" (UniqueName: \"kubernetes.io/projected/9b189f01-3984-4478-a7cf-9aa1e0d36a43-kube-api-access-dhz6n\") pod \"lws-controller-manager-56954ff8c6-khqvt\" (UID: \"9b189f01-3984-4478-a7cf-9aa1e0d36a43\") " pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.202989 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.202959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:32.343542 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:32.343464    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"]
Jul 08 02:48:32.347145 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:48:32.347112    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b189f01_3984_4478_a7cf_9aa1e0d36a43.slice/crio-0132eeab7ece3fcbd44e69d1d0229dd81fd69f05100af198379dcca0d8acba58 WatchSource:0}: Error finding container 0132eeab7ece3fcbd44e69d1d0229dd81fd69f05100af198379dcca0d8acba58: Status 404 returned error can't find the container with id 0132eeab7ece3fcbd44e69d1d0229dd81fd69f05100af198379dcca0d8acba58
Jul 08 02:48:33.329212 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:33.329176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt" event={"ID":"9b189f01-3984-4478-a7cf-9aa1e0d36a43","Type":"ContainerStarted","Data":"0132eeab7ece3fcbd44e69d1d0229dd81fd69f05100af198379dcca0d8acba58"}
Jul 08 02:48:35.245758 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.245728    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"]
Jul 08 02:48:35.272396 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.272359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"]
Jul 08 02:48:35.272572 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.272497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.275623 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.275599    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 08 02:48:35.275766 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.275709    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 08 02:48:35.278537 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.278477    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7n7w2\""
Jul 08 02:48:35.278537 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.278486    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 08 02:48:35.279001 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.278980    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 08 02:48:35.343387 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.343349    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt" event={"ID":"9b189f01-3984-4478-a7cf-9aa1e0d36a43","Type":"ContainerStarted","Data":"c8f2970e333d87f845d8c4a7c15e44c7418c10cccae38e5ebadbf06e7dfe872a"}
Jul 08 02:48:35.343556 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.343468    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:35.371930 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.371879    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt" podStartSLOduration=2.4494030589999998 podStartE2EDuration="4.371860026s" totalImagesPullingTime="1.922456967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:48:35.370294478 +0000 UTC m=+503.298156097" watchObservedRunningTime="2026-07-08 02:48:35.371860026 +0000 UTC m=+503.299721626"
Jul 08 02:48:35.379098 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.379064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjw4j\" (UniqueName: \"kubernetes.io/projected/bc676c0b-06d2-4c50-b9cc-075164db6a0f-kube-api-access-jjw4j\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.379221 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.379113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc676c0b-06d2-4c50-b9cc-075164db6a0f-tmp\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.379221 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.379135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc676c0b-06d2-4c50-b9cc-075164db6a0f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.379221 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.379203    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc676c0b-06d2-4c50-b9cc-075164db6a0f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.379346 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.379244    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc676c0b-06d2-4c50-b9cc-075164db6a0f-manifests\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.480401 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.480354    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjw4j\" (UniqueName: \"kubernetes.io/projected/bc676c0b-06d2-4c50-b9cc-075164db6a0f-kube-api-access-jjw4j\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.480584 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.480559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc676c0b-06d2-4c50-b9cc-075164db6a0f-tmp\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.480627 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.480613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc676c0b-06d2-4c50-b9cc-075164db6a0f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.480673 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.480655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc676c0b-06d2-4c50-b9cc-075164db6a0f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.480805 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.480781    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc676c0b-06d2-4c50-b9cc-075164db6a0f-manifests\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.481110 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.481089    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc676c0b-06d2-4c50-b9cc-075164db6a0f-manifests\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.483582 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.483555    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc676c0b-06d2-4c50-b9cc-075164db6a0f-tmp\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.483775 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.483753    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc676c0b-06d2-4c50-b9cc-075164db6a0f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.483863 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.483755    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc676c0b-06d2-4c50-b9cc-075164db6a0f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.491459 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.491430    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjw4j\" (UniqueName: \"kubernetes.io/projected/bc676c0b-06d2-4c50-b9cc-075164db6a0f-kube-api-access-jjw4j\") pod \"opendatahub-operator-controller-manager-6fbb74d9dd-w285s\" (UID: \"bc676c0b-06d2-4c50-b9cc-075164db6a0f\") " pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.583255 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.583152    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:35.747069 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:35.747036    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"]
Jul 08 02:48:35.750228 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:48:35.750200    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc676c0b_06d2_4c50_b9cc_075164db6a0f.slice/crio-cf83aa76cd019e02eae82ceb705380a85d3f6eb329de4aef7268e838d209f3d2 WatchSource:0}: Error finding container cf83aa76cd019e02eae82ceb705380a85d3f6eb329de4aef7268e838d209f3d2: Status 404 returned error can't find the container with id cf83aa76cd019e02eae82ceb705380a85d3f6eb329de4aef7268e838d209f3d2
Jul 08 02:48:36.348608 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:36.348575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s" event={"ID":"bc676c0b-06d2-4c50-b9cc-075164db6a0f","Type":"ContainerStarted","Data":"cf83aa76cd019e02eae82ceb705380a85d3f6eb329de4aef7268e838d209f3d2"}
Jul 08 02:48:39.362563 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:39.362525    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s" event={"ID":"bc676c0b-06d2-4c50-b9cc-075164db6a0f","Type":"ContainerStarted","Data":"adfc8f65a8f38b300f775f92f21de5f46de63563634cb368050ff149b0d32e41"}
Jul 08 02:48:40.368277 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:40.368189    2558 generic.go:356] "Generic (PLEG): container finished" podID="bc676c0b-06d2-4c50-b9cc-075164db6a0f" containerID="adfc8f65a8f38b300f775f92f21de5f46de63563634cb368050ff149b0d32e41" exitCode=0
Jul 08 02:48:40.368638 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:40.368275    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s" event={"ID":"bc676c0b-06d2-4c50-b9cc-075164db6a0f","Type":"ContainerDied","Data":"adfc8f65a8f38b300f775f92f21de5f46de63563634cb368050ff149b0d32e41"}
Jul 08 02:48:41.374223 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:41.374183    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s" event={"ID":"bc676c0b-06d2-4c50-b9cc-075164db6a0f","Type":"ContainerStarted","Data":"0bbaeafa16ba2e93c2470612daf7886a734c16205759a7c08a6fd2cd3f64f31e"}
Jul 08 02:48:41.374597 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:41.374336    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:41.397790 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:41.397738    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s" podStartSLOduration=1.756100486 podStartE2EDuration="6.397720939s" totalImagesPullingTime="3.554045703s" totalInitContainerRuntime="1.08757475s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:48:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:48:41.395727744 +0000 UTC m=+509.323589355" watchObservedRunningTime="2026-07-08 02:48:41.397720939 +0000 UTC m=+509.325582549"
Jul 08 02:48:46.350959 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:46.350933    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56954ff8c6-khqvt"
Jul 08 02:48:52.380805 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:52.380769    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fbb74d9dd-w285s"
Jul 08 02:48:54.826955 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.826913    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"]
Jul 08 02:48:54.835002 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.834980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:54.838924 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.838900    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 02:48:54.839063 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.838954    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w26jf\""
Jul 08 02:48:54.839371 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.839355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 08 02:48:54.839902 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.839886    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 08 02:48:54.839988 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.839924    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 02:48:54.847665 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.847639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"]
Jul 08 02:48:54.939395 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.939363    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q875\" (UniqueName: \"kubernetes.io/projected/7311b400-ba9c-45bc-afed-b19be4974ca6-kube-api-access-4q875\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:54.939589 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.939422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7311b400-ba9c-45bc-afed-b19be4974ca6-tls-certs\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:54.939589 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:54.939444    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7311b400-ba9c-45bc-afed-b19be4974ca6-tmp\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.040667 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.040627    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q875\" (UniqueName: \"kubernetes.io/projected/7311b400-ba9c-45bc-afed-b19be4974ca6-kube-api-access-4q875\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.040900 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.040693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7311b400-ba9c-45bc-afed-b19be4974ca6-tls-certs\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.040900 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.040720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7311b400-ba9c-45bc-afed-b19be4974ca6-tmp\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.043298 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.043258    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7311b400-ba9c-45bc-afed-b19be4974ca6-tmp\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.043411 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.043354    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7311b400-ba9c-45bc-afed-b19be4974ca6-tls-certs\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.054226 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.054196    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q875\" (UniqueName: \"kubernetes.io/projected/7311b400-ba9c-45bc-afed-b19be4974ca6-kube-api-access-4q875\") pod \"kube-auth-proxy-dcfc578f7-7vslg\" (UID: \"7311b400-ba9c-45bc-afed-b19be4974ca6\") " pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.144457 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.144367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"
Jul 08 02:48:55.279677 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.279643    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg"]
Jul 08 02:48:55.283456 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:48:55.283433    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7311b400_ba9c_45bc_afed_b19be4974ca6.slice/crio-ade245ca189d058e88bf3731c34429145d7298cab8458d00a46a49d9b260b1b8 WatchSource:0}: Error finding container ade245ca189d058e88bf3731c34429145d7298cab8458d00a46a49d9b260b1b8: Status 404 returned error can't find the container with id ade245ca189d058e88bf3731c34429145d7298cab8458d00a46a49d9b260b1b8
Jul 08 02:48:55.430397 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:55.430366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg" event={"ID":"7311b400-ba9c-45bc-afed-b19be4974ca6","Type":"ContainerStarted","Data":"ade245ca189d058e88bf3731c34429145d7298cab8458d00a46a49d9b260b1b8"}
Jul 08 02:48:58.447592 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:58.447559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg" event={"ID":"7311b400-ba9c-45bc-afed-b19be4974ca6","Type":"ContainerStarted","Data":"4153c7121cb5cc21991d0de7888746a2a516617a2cba87deba71c4539006b1a6"}
Jul 08 02:48:58.468961 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:48:58.468894    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-dcfc578f7-7vslg" podStartSLOduration=1.8194361780000001 podStartE2EDuration="4.468872213s" totalImagesPullingTime="2.649436035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:48:58.467435597 +0000 UTC m=+526.395297208" watchObservedRunningTime="2026-07-08 02:48:58.468872213 +0000 UTC m=+526.396733824"
Jul 08 02:50:11.200482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.200442    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"]
Jul 08 02:50:11.203830 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.203813    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.207404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.207377    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 02:50:11.207404 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.207396    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vrdv4\""
Jul 08 02:50:11.207617 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.207428    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 02:50:11.212644 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.212622    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"]
Jul 08 02:50:11.318162 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.318122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.318335 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.318169    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqng\" (UniqueName: \"kubernetes.io/projected/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-kube-api-access-grqng\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.318335 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.318269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.418647 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.418612    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqng\" (UniqueName: \"kubernetes.io/projected/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-kube-api-access-grqng\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.418793 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.418668    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.418793 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.418697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.419047 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.419030    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.419127 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.419097    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.427452 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.427428    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqng\" (UniqueName: \"kubernetes.io/projected/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-kube-api-access-grqng\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.513526 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.513430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:11.641952 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.641911    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"]
Jul 08 02:50:11.644792 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:50:11.644764    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d3c4a0_dd94_454d_bf03_f82ff578ce24.slice/crio-91a52b6f6dba9e8b3e7c2a42bc2ee4399a7fb644c7c673ac95df70cf7b2dcac5 WatchSource:0}: Error finding container 91a52b6f6dba9e8b3e7c2a42bc2ee4399a7fb644c7c673ac95df70cf7b2dcac5: Status 404 returned error can't find the container with id 91a52b6f6dba9e8b3e7c2a42bc2ee4399a7fb644c7c673ac95df70cf7b2dcac5
Jul 08 02:50:11.736342 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.736303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv" event={"ID":"e5d3c4a0-dd94-454d-bf03-f82ff578ce24","Type":"ContainerStarted","Data":"29f31e02b7c1ae311b726d799da6d4535f0c85528e86158b64a7b9cd7bba6c0e"}
Jul 08 02:50:11.736534 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.736354    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv" event={"ID":"e5d3c4a0-dd94-454d-bf03-f82ff578ce24","Type":"ContainerStarted","Data":"91a52b6f6dba9e8b3e7c2a42bc2ee4399a7fb644c7c673ac95df70cf7b2dcac5"}
Jul 08 02:50:11.996834 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:11.996798    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"]
Jul 08 02:50:12.000053 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.000037    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.008603 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.008579    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"]
Jul 08 02:50:12.123887 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.123824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.124113 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.123898    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p68jk\" (UniqueName: \"kubernetes.io/projected/8deb1abe-426b-4c70-86e4-1b74b508b12e-kube-api-access-p68jk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.124113 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.124020    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.224696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.224663    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.224696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.224697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p68jk\" (UniqueName: \"kubernetes.io/projected/8deb1abe-426b-4c70-86e4-1b74b508b12e-kube-api-access-p68jk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.225243 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.224762    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.225243 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.225149    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.225243 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.225188    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.233029 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.233008    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p68jk\" (UniqueName: \"kubernetes.io/projected/8deb1abe-426b-4c70-86e4-1b74b508b12e-kube-api-access-p68jk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.309064 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.308987    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:12.434477 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.434447    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"]
Jul 08 02:50:12.436726 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:50:12.436696    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8deb1abe_426b_4c70_86e4_1b74b508b12e.slice/crio-d6df688f6d31b8c886790115c5972d1f2610dffab2ec5ca7fc5ad4a73c824aab WatchSource:0}: Error finding container d6df688f6d31b8c886790115c5972d1f2610dffab2ec5ca7fc5ad4a73c824aab: Status 404 returned error can't find the container with id d6df688f6d31b8c886790115c5972d1f2610dffab2ec5ca7fc5ad4a73c824aab
Jul 08 02:50:12.487641 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.487612    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:50:12.488129 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.488105    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:50:12.742060 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.742017    2558 generic.go:356] "Generic (PLEG): container finished" podID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerID="29f31e02b7c1ae311b726d799da6d4535f0c85528e86158b64a7b9cd7bba6c0e" exitCode=0
Jul 08 02:50:12.742238 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.742104    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv" event={"ID":"e5d3c4a0-dd94-454d-bf03-f82ff578ce24","Type":"ContainerDied","Data":"29f31e02b7c1ae311b726d799da6d4535f0c85528e86158b64a7b9cd7bba6c0e"}
Jul 08 02:50:12.743609 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.743591    2558 generic.go:356] "Generic (PLEG): container finished" podID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerID="ab31c4cd21c2c8c1c9579d8fd28137849444c77762b55e390d394e1e0a05bde6" exitCode=0
Jul 08 02:50:12.743684 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.743619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75" event={"ID":"8deb1abe-426b-4c70-86e4-1b74b508b12e","Type":"ContainerDied","Data":"ab31c4cd21c2c8c1c9579d8fd28137849444c77762b55e390d394e1e0a05bde6"}
Jul 08 02:50:12.743684 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.743661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75" event={"ID":"8deb1abe-426b-4c70-86e4-1b74b508b12e","Type":"ContainerStarted","Data":"d6df688f6d31b8c886790115c5972d1f2610dffab2ec5ca7fc5ad4a73c824aab"}
Jul 08 02:50:12.804264 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.804231    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"]
Jul 08 02:50:12.807405 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.807389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:12.815231 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.815208    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"]
Jul 08 02:50:12.930340 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.930298    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kccfb\" (UniqueName: \"kubernetes.io/projected/4a98cf78-aa55-4048-bdad-f35e1827893e-kube-api-access-kccfb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:12.930493 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.930389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:12.930493 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:12.930420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.031775 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.031689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kccfb\" (UniqueName: \"kubernetes.io/projected/4a98cf78-aa55-4048-bdad-f35e1827893e-kube-api-access-kccfb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.031775 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.031734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.031775 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.031753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.032081 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.032066    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.032134 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.032114    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.039958 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.039923    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kccfb\" (UniqueName: \"kubernetes.io/projected/4a98cf78-aa55-4048-bdad-f35e1827893e-kube-api-access-kccfb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.116396 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.116363    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:13.202869 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.202811    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"]
Jul 08 02:50:13.206380 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.206357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.215873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.215820    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"]
Jul 08 02:50:13.246149 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.246126    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"]
Jul 08 02:50:13.248428 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:50:13.248402    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a98cf78_aa55_4048_bdad_f35e1827893e.slice/crio-c1ee9252070a3fb79e4ff16327dd95dbb3bbb53fdcee864510325603402f9bf5 WatchSource:0}: Error finding container c1ee9252070a3fb79e4ff16327dd95dbb3bbb53fdcee864510325603402f9bf5: Status 404 returned error can't find the container with id c1ee9252070a3fb79e4ff16327dd95dbb3bbb53fdcee864510325603402f9bf5
Jul 08 02:50:13.334472 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.334434    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.334642 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.334480    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cksm\" (UniqueName: \"kubernetes.io/projected/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-kube-api-access-5cksm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.334642 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.334533    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.435158 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.435107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cksm\" (UniqueName: \"kubernetes.io/projected/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-kube-api-access-5cksm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.435314 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.435170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.435314 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.435262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.435606 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.435580    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.435696 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.435645    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.444403 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.444381    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cksm\" (UniqueName: \"kubernetes.io/projected/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-kube-api-access-5cksm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.519899 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.519868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:13.753092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.753064    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerID="597087c1575cf95254c3cd6e3b2d0ef5b3340aa9b1ad59374039f58437127745" exitCode=0
Jul 08 02:50:13.753214 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.753146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn" event={"ID":"4a98cf78-aa55-4048-bdad-f35e1827893e","Type":"ContainerDied","Data":"597087c1575cf95254c3cd6e3b2d0ef5b3340aa9b1ad59374039f58437127745"}
Jul 08 02:50:13.753214 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.753179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn" event={"ID":"4a98cf78-aa55-4048-bdad-f35e1827893e","Type":"ContainerStarted","Data":"c1ee9252070a3fb79e4ff16327dd95dbb3bbb53fdcee864510325603402f9bf5"}
Jul 08 02:50:13.756598 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.756574    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75" event={"ID":"8deb1abe-426b-4c70-86e4-1b74b508b12e","Type":"ContainerStarted","Data":"3c3ac7386d563a6e4c946bcd9628242f187684d047bd09b7e2eb997a40fa2361"}
Jul 08 02:50:13.781015 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:13.780988    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"]
Jul 08 02:50:13.783510 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:50:13.783481    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5fc2480_3c0d_4926_a2ab_f402e31dc2a0.slice/crio-85cb5c7d47e6b5095aeed2ae13e120606155fe1399df4d59fd79ab8297b0d0f1 WatchSource:0}: Error finding container 85cb5c7d47e6b5095aeed2ae13e120606155fe1399df4d59fd79ab8297b0d0f1: Status 404 returned error can't find the container with id 85cb5c7d47e6b5095aeed2ae13e120606155fe1399df4d59fd79ab8297b0d0f1
Jul 08 02:50:14.762182 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.762141    2558 generic.go:356] "Generic (PLEG): container finished" podID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerID="d05d3834599c00899a4661cacf5e2bb3cb8eb59a882bd8a00e30c70273c96a58" exitCode=0
Jul 08 02:50:14.762621 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.762187    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6" event={"ID":"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0","Type":"ContainerDied","Data":"d05d3834599c00899a4661cacf5e2bb3cb8eb59a882bd8a00e30c70273c96a58"}
Jul 08 02:50:14.762621 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.762263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6" event={"ID":"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0","Type":"ContainerStarted","Data":"85cb5c7d47e6b5095aeed2ae13e120606155fe1399df4d59fd79ab8297b0d0f1"}
Jul 08 02:50:14.764259 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.764235    2558 generic.go:356] "Generic (PLEG): container finished" podID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerID="5bfabd9642e1c23b99d182e9be8a9e1595e8e94368f019cc1925cbf497682544" exitCode=0
Jul 08 02:50:14.764362 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.764317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv" event={"ID":"e5d3c4a0-dd94-454d-bf03-f82ff578ce24","Type":"ContainerDied","Data":"5bfabd9642e1c23b99d182e9be8a9e1595e8e94368f019cc1925cbf497682544"}
Jul 08 02:50:14.766227 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.766204    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerID="e9672222fbd403d6b177faa58fc257599f8f73b015272ec19ce1fb3ba55d319d" exitCode=0
Jul 08 02:50:14.766320 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.766259    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn" event={"ID":"4a98cf78-aa55-4048-bdad-f35e1827893e","Type":"ContainerDied","Data":"e9672222fbd403d6b177faa58fc257599f8f73b015272ec19ce1fb3ba55d319d"}
Jul 08 02:50:14.768638 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.768615    2558 generic.go:356] "Generic (PLEG): container finished" podID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerID="3c3ac7386d563a6e4c946bcd9628242f187684d047bd09b7e2eb997a40fa2361" exitCode=0
Jul 08 02:50:14.768715 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:14.768692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75" event={"ID":"8deb1abe-426b-4c70-86e4-1b74b508b12e","Type":"ContainerDied","Data":"3c3ac7386d563a6e4c946bcd9628242f187684d047bd09b7e2eb997a40fa2361"}
Jul 08 02:50:15.775667 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.775632    2558 generic.go:356] "Generic (PLEG): container finished" podID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerID="88360748d383ca745db4c2369e254d099953a6076dce844ae6f6129969e4ddc5" exitCode=0
Jul 08 02:50:15.776112 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.775724    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6" event={"ID":"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0","Type":"ContainerDied","Data":"88360748d383ca745db4c2369e254d099953a6076dce844ae6f6129969e4ddc5"}
Jul 08 02:50:15.777988 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.777969    2558 generic.go:356] "Generic (PLEG): container finished" podID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerID="fb943d4e5a32601d277949b996e709ad6b7aa41ad8ddb35c78f28d0545a0dac3" exitCode=0
Jul 08 02:50:15.778083 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.778058    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv" event={"ID":"e5d3c4a0-dd94-454d-bf03-f82ff578ce24","Type":"ContainerDied","Data":"fb943d4e5a32601d277949b996e709ad6b7aa41ad8ddb35c78f28d0545a0dac3"}
Jul 08 02:50:15.780123 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.780100    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerID="570e9073c75a7daf882ae758f431d00dad7e7ba4ab04a20ac2072a5cf65de611" exitCode=0
Jul 08 02:50:15.780216 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.780173    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn" event={"ID":"4a98cf78-aa55-4048-bdad-f35e1827893e","Type":"ContainerDied","Data":"570e9073c75a7daf882ae758f431d00dad7e7ba4ab04a20ac2072a5cf65de611"}
Jul 08 02:50:15.782201 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.782181    2558 generic.go:356] "Generic (PLEG): container finished" podID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerID="aaab0938ab91ed4096d3b00c00f3e243d6c31d05205c473cfeae5a25338be0df" exitCode=0
Jul 08 02:50:15.782285 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:15.782241    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75" event={"ID":"8deb1abe-426b-4c70-86e4-1b74b508b12e","Type":"ContainerDied","Data":"aaab0938ab91ed4096d3b00c00f3e243d6c31d05205c473cfeae5a25338be0df"}
Jul 08 02:50:16.789878 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.789820    2558 generic.go:356] "Generic (PLEG): container finished" podID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerID="3a40136666c60ec698d6825aec35957e2cd34524616f06e5fd2ea71226f490c3" exitCode=0
Jul 08 02:50:16.792319 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.789911    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6" event={"ID":"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0","Type":"ContainerDied","Data":"3a40136666c60ec698d6825aec35957e2cd34524616f06e5fd2ea71226f490c3"}
Jul 08 02:50:16.917059 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.917034    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:16.960115 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.960095    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:16.965120 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.965094    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grqng\" (UniqueName: \"kubernetes.io/projected/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-kube-api-access-grqng\") pod \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") "
Jul 08 02:50:16.965240 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.965174    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-util\") pod \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") "
Jul 08 02:50:16.965240 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.965221    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kccfb\" (UniqueName: \"kubernetes.io/projected/4a98cf78-aa55-4048-bdad-f35e1827893e-kube-api-access-kccfb\") pod \"4a98cf78-aa55-4048-bdad-f35e1827893e\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") "
Jul 08 02:50:16.965349 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.965257    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-bundle\") pod \"4a98cf78-aa55-4048-bdad-f35e1827893e\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") "
Jul 08 02:50:16.965349 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.965291    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-util\") pod \"4a98cf78-aa55-4048-bdad-f35e1827893e\" (UID: \"4a98cf78-aa55-4048-bdad-f35e1827893e\") "
Jul 08 02:50:16.965349 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.965312    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-bundle\") pod \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\" (UID: \"e5d3c4a0-dd94-454d-bf03-f82ff578ce24\") "
Jul 08 02:50:16.966250 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.966197    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-bundle" (OuterVolumeSpecName: "bundle") pod "e5d3c4a0-dd94-454d-bf03-f82ff578ce24" (UID: "e5d3c4a0-dd94-454d-bf03-f82ff578ce24"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:16.966250 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.966222    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-bundle" (OuterVolumeSpecName: "bundle") pod "4a98cf78-aa55-4048-bdad-f35e1827893e" (UID: "4a98cf78-aa55-4048-bdad-f35e1827893e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:16.967699 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.967675    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-kube-api-access-grqng" (OuterVolumeSpecName: "kube-api-access-grqng") pod "e5d3c4a0-dd94-454d-bf03-f82ff578ce24" (UID: "e5d3c4a0-dd94-454d-bf03-f82ff578ce24"). InnerVolumeSpecName "kube-api-access-grqng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:50:16.967817 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.967676    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a98cf78-aa55-4048-bdad-f35e1827893e-kube-api-access-kccfb" (OuterVolumeSpecName: "kube-api-access-kccfb") pod "4a98cf78-aa55-4048-bdad-f35e1827893e" (UID: "4a98cf78-aa55-4048-bdad-f35e1827893e"). InnerVolumeSpecName "kube-api-access-kccfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:50:16.970974 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.970955    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:16.971471 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:16.971447    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-util" (OuterVolumeSpecName: "util") pod "4a98cf78-aa55-4048-bdad-f35e1827893e" (UID: "4a98cf78-aa55-4048-bdad-f35e1827893e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:17.053773 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.053695    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-util" (OuterVolumeSpecName: "util") pod "e5d3c4a0-dd94-454d-bf03-f82ff578ce24" (UID: "e5d3c4a0-dd94-454d-bf03-f82ff578ce24"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:17.065678 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065647    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-bundle\") pod \"8deb1abe-426b-4c70-86e4-1b74b508b12e\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") "
Jul 08 02:50:17.065814 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065687    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p68jk\" (UniqueName: \"kubernetes.io/projected/8deb1abe-426b-4c70-86e4-1b74b508b12e-kube-api-access-p68jk\") pod \"8deb1abe-426b-4c70-86e4-1b74b508b12e\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") "
Jul 08 02:50:17.065814 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065718    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-util\") pod \"8deb1abe-426b-4c70-86e4-1b74b508b12e\" (UID: \"8deb1abe-426b-4c70-86e4-1b74b508b12e\") "
Jul 08 02:50:17.065943 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065927    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.065982 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065949    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a98cf78-aa55-4048-bdad-f35e1827893e-util\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.065982 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065963    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.065982 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065978    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grqng\" (UniqueName: \"kubernetes.io/projected/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-kube-api-access-grqng\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.066140 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.065993    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5d3c4a0-dd94-454d-bf03-f82ff578ce24-util\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.066140 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.066005    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kccfb\" (UniqueName: \"kubernetes.io/projected/4a98cf78-aa55-4048-bdad-f35e1827893e-kube-api-access-kccfb\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.066200 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.066166    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-bundle" (OuterVolumeSpecName: "bundle") pod "8deb1abe-426b-4c70-86e4-1b74b508b12e" (UID: "8deb1abe-426b-4c70-86e4-1b74b508b12e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:17.067795 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.067773    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8deb1abe-426b-4c70-86e4-1b74b508b12e-kube-api-access-p68jk" (OuterVolumeSpecName: "kube-api-access-p68jk") pod "8deb1abe-426b-4c70-86e4-1b74b508b12e" (UID: "8deb1abe-426b-4c70-86e4-1b74b508b12e"). InnerVolumeSpecName "kube-api-access-p68jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:50:17.071220 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.071199    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-util" (OuterVolumeSpecName: "util") pod "8deb1abe-426b-4c70-86e4-1b74b508b12e" (UID: "8deb1abe-426b-4c70-86e4-1b74b508b12e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:17.166750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.166718    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-util\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.166750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.166744    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8deb1abe-426b-4c70-86e4-1b74b508b12e-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.166750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.166753    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p68jk\" (UniqueName: \"kubernetes.io/projected/8deb1abe-426b-4c70-86e4-1b74b508b12e-kube-api-access-p68jk\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:17.796637 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.796610    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv"
Jul 08 02:50:17.796637 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.796613    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv" event={"ID":"e5d3c4a0-dd94-454d-bf03-f82ff578ce24","Type":"ContainerDied","Data":"91a52b6f6dba9e8b3e7c2a42bc2ee4399a7fb644c7c673ac95df70cf7b2dcac5"}
Jul 08 02:50:17.797174 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.796657    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91a52b6f6dba9e8b3e7c2a42bc2ee4399a7fb644c7c673ac95df70cf7b2dcac5"
Jul 08 02:50:17.798585 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.798556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn" event={"ID":"4a98cf78-aa55-4048-bdad-f35e1827893e","Type":"ContainerDied","Data":"c1ee9252070a3fb79e4ff16327dd95dbb3bbb53fdcee864510325603402f9bf5"}
Jul 08 02:50:17.798585 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.798583    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn"
Jul 08 02:50:17.798820 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.798606    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1ee9252070a3fb79e4ff16327dd95dbb3bbb53fdcee864510325603402f9bf5"
Jul 08 02:50:17.800607 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.800575    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75"
Jul 08 02:50:17.800607 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.800599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75" event={"ID":"8deb1abe-426b-4c70-86e4-1b74b508b12e","Type":"ContainerDied","Data":"d6df688f6d31b8c886790115c5972d1f2610dffab2ec5ca7fc5ad4a73c824aab"}
Jul 08 02:50:17.800802 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.800637    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6df688f6d31b8c886790115c5972d1f2610dffab2ec5ca7fc5ad4a73c824aab"
Jul 08 02:50:17.931886 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.931861    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:17.971978 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.971950    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cksm\" (UniqueName: \"kubernetes.io/projected/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-kube-api-access-5cksm\") pod \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") "
Jul 08 02:50:17.971978 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.971985    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-bundle\") pod \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") "
Jul 08 02:50:17.972189 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.972009    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-util\") pod \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\" (UID: \"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0\") "
Jul 08 02:50:17.972520 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.972496    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-bundle" (OuterVolumeSpecName: "bundle") pod "d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" (UID: "d5fc2480-3c0d-4926-a2ab-f402e31dc2a0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:17.974313 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.974289    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-kube-api-access-5cksm" (OuterVolumeSpecName: "kube-api-access-5cksm") pod "d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" (UID: "d5fc2480-3c0d-4926-a2ab-f402e31dc2a0"). InnerVolumeSpecName "kube-api-access-5cksm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:50:17.977221 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:17.977198    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-util" (OuterVolumeSpecName: "util") pod "d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" (UID: "d5fc2480-3c0d-4926-a2ab-f402e31dc2a0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 02:50:18.073337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:18.073240    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-util\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:18.073337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:18.073279    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cksm\" (UniqueName: \"kubernetes.io/projected/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-kube-api-access-5cksm\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:18.073337 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:18.073291    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fc2480-3c0d-4926-a2ab-f402e31dc2a0-bundle\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:50:18.807021 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:18.806986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6" event={"ID":"d5fc2480-3c0d-4926-a2ab-f402e31dc2a0","Type":"ContainerDied","Data":"85cb5c7d47e6b5095aeed2ae13e120606155fe1399df4d59fd79ab8297b0d0f1"}
Jul 08 02:50:18.807021 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:18.807009    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6"
Jul 08 02:50:18.807538 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:18.807037    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85cb5c7d47e6b5095aeed2ae13e120606155fe1399df4d59fd79ab8297b0d0f1"
Jul 08 02:50:47.033078 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.032996    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"]
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033239    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033249    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033258    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033263    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033276    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033281    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033288    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033293    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033300    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033304    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033315    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033321    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033327    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033332    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033338    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033343    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033350    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033356    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="util"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033362    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033366    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033371    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033376    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="pull"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033382    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033386    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033423    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8deb1abe-426b-4c70-86e4-1b74b508b12e" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033430    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a98cf78-aa55-4048-bdad-f35e1827893e" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033437    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5fc2480-3c0d-4926-a2ab-f402e31dc2a0" containerName="extract"
Jul 08 02:50:47.033547 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.033442    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d3c4a0-dd94-454d-bf03-f82ff578ce24" containerName="extract"
Jul 08 02:50:47.036355 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.036339    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.039060 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.039036    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 08 02:50:47.039060 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.039057    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 02:50:47.039238 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.039036    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 08 02:50:47.039318 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.039300    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vrdv4\""
Jul 08 02:50:47.040032 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.040017    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 02:50:47.049245 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.049219    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"]
Jul 08 02:50:47.086229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.086196    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6178861-a443-4fd3-bd5f-213be928d672-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.086229 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.086225    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92jrf\" (UniqueName: \"kubernetes.io/projected/b6178861-a443-4fd3-bd5f-213be928d672-kube-api-access-92jrf\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.086421 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.086293    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6178861-a443-4fd3-bd5f-213be928d672-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.186931 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.186893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6178861-a443-4fd3-bd5f-213be928d672-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.186931 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.186932    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92jrf\" (UniqueName: \"kubernetes.io/projected/b6178861-a443-4fd3-bd5f-213be928d672-kube-api-access-92jrf\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.187190 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.186990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6178861-a443-4fd3-bd5f-213be928d672-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.187651 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.187628    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6178861-a443-4fd3-bd5f-213be928d672-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.189434 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.189415    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6178861-a443-4fd3-bd5f-213be928d672-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.199043 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.199021    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92jrf\" (UniqueName: \"kubernetes.io/projected/b6178861-a443-4fd3-bd5f-213be928d672-kube-api-access-92jrf\") pod \"kuadrant-console-plugin-66b5774985-mlqxn\" (UID: \"b6178861-a443-4fd3-bd5f-213be928d672\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.345286 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.345181    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"
Jul 08 02:50:47.493621 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.493591    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn"]
Jul 08 02:50:47.496676 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:50:47.496635    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6178861_a443_4fd3_bd5f_213be928d672.slice/crio-8799e1fcfcf0178e12a216f1af3cc881b91576054430e2a266bdeccdda62136b WatchSource:0}: Error finding container 8799e1fcfcf0178e12a216f1af3cc881b91576054430e2a266bdeccdda62136b: Status 404 returned error can't find the container with id 8799e1fcfcf0178e12a216f1af3cc881b91576054430e2a266bdeccdda62136b
Jul 08 02:50:47.932437 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:50:47.932403    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn" event={"ID":"b6178861-a443-4fd3-bd5f-213be928d672","Type":"ContainerStarted","Data":"8799e1fcfcf0178e12a216f1af3cc881b91576054430e2a266bdeccdda62136b"}
Jul 08 02:51:09.050222 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:09.050141    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn" event={"ID":"b6178861-a443-4fd3-bd5f-213be928d672","Type":"ContainerStarted","Data":"48514e9ae376384c34bee7316b3f2825af2370216a79f069cbb8f2bc2f374d05"}
Jul 08 02:51:09.067795 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:09.067741    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mlqxn" podStartSLOduration=0.894686161 podStartE2EDuration="22.067722925s" totalImagesPullingTime="21.173036764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:51:09.065995126 +0000 UTC m=+656.993856736" watchObservedRunningTime="2026-07-08 02:51:09.067722925 +0000 UTC m=+656.995584534"
Jul 08 02:51:15.256960 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.256925    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d7bf7d9f7-zswmz"]
Jul 08 02:51:15.372335 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.372295    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7bf7d9f7-zswmz"]
Jul 08 02:51:15.372486 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.372399    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.375682 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.375657    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 02:51:15.375833 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.375686    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 02:51:15.375833 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.375692    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 02:51:15.376061 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.376045    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 02:51:15.376187 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.376168    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 02:51:15.377056 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.377038    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 02:51:15.377164 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.377071    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 02:51:15.377164 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.377095    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ws27b\""
Jul 08 02:51:15.382802 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.382782    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-oauth-config\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433586    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-trusted-ca-bundle\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433629    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-service-ca\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433662    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-oauth-serving-cert\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433711    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-config\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433766    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-serving-cert\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.434873 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.433806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92bbj\" (UniqueName: \"kubernetes.io/projected/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-kube-api-access-92bbj\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.534815 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-oauth-serving-cert\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.534815 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534727    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-config\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.534815 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-serving-cert\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.534815 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92bbj\" (UniqueName: \"kubernetes.io/projected/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-kube-api-access-92bbj\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.535262 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-oauth-config\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.535262 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-trusted-ca-bundle\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.535262 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.534937    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-service-ca\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.535500 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.535470    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-oauth-serving-cert\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.535686 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.535667    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-service-ca\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.535858 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.535819    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-trusted-ca-bundle\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.536077 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.536059    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-config\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.537420 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.537388    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-oauth-config\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.537497 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.537472    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-console-serving-cert\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.543300 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.543280    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92bbj\" (UniqueName: \"kubernetes.io/projected/d4ca526e-6dfe-4f1c-a9b5-27c32fdde440-kube-api-access-92bbj\") pod \"console-d7bf7d9f7-zswmz\" (UID: \"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440\") " pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.682285 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.682245    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:15.831277 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:15.831253    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7bf7d9f7-zswmz"]
Jul 08 02:51:15.832804 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:51:15.832772    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ca526e_6dfe_4f1c_a9b5_27c32fdde440.slice/crio-eff70657a2045a6fe74f5f82b3c1d277ea83cc0dce96d07849c84c12d972816b WatchSource:0}: Error finding container eff70657a2045a6fe74f5f82b3c1d277ea83cc0dce96d07849c84c12d972816b: Status 404 returned error can't find the container with id eff70657a2045a6fe74f5f82b3c1d277ea83cc0dce96d07849c84c12d972816b
Jul 08 02:51:16.083478 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:16.083389    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7bf7d9f7-zswmz" event={"ID":"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440","Type":"ContainerStarted","Data":"8d44327590fd95afc92612c9eccbb40d519569390fa4138e054d1df53b78808f"}
Jul 08 02:51:16.083478 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:16.083459    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7bf7d9f7-zswmz" event={"ID":"d4ca526e-6dfe-4f1c-a9b5-27c32fdde440","Type":"ContainerStarted","Data":"eff70657a2045a6fe74f5f82b3c1d277ea83cc0dce96d07849c84c12d972816b"}
Jul 08 02:51:16.103217 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:16.103169    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d7bf7d9f7-zswmz" podStartSLOduration=1.103153461 podStartE2EDuration="1.103153461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:51:16.101192358 +0000 UTC m=+664.029053985" watchObservedRunningTime="2026-07-08 02:51:16.103153461 +0000 UTC m=+664.031015072"
Jul 08 02:51:25.683379 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:25.683329    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:25.683783 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:25.683403    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:25.687985 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:25.687962    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:26.133399 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:26.133319    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d7bf7d9f7-zswmz"
Jul 08 02:51:39.191284 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.191247    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vqlmz"]
Jul 08 02:51:39.368943 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.368902    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vqlmz"]
Jul 08 02:51:39.368943 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.368940    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlzzn"]
Jul 08 02:51:39.369194 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.369038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:39.371660 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.371635    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-648bj\""
Jul 08 02:51:39.380645 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.380626    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlzzn"]
Jul 08 02:51:39.380747 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.380707    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:51:39.434821 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.434784    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrgc6\" (UniqueName: \"kubernetes.io/projected/9dfd43c1-1a94-4cd7-afc7-505430812ae2-kube-api-access-zrgc6\") pod \"authorino-75d556bd77-wlzzn\" (UID: \"9dfd43c1-1a94-4cd7-afc7-505430812ae2\") " pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:51:39.435008 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.434849    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9vwn\" (UniqueName: \"kubernetes.io/projected/5a733006-b68c-4968-b5f5-8fc583b72a47-kube-api-access-w9vwn\") pod \"authorino-58cd955675-vqlmz\" (UID: \"5a733006-b68c-4968-b5f5-8fc583b72a47\") " pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:39.536115 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.536033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrgc6\" (UniqueName: \"kubernetes.io/projected/9dfd43c1-1a94-4cd7-afc7-505430812ae2-kube-api-access-zrgc6\") pod \"authorino-75d556bd77-wlzzn\" (UID: \"9dfd43c1-1a94-4cd7-afc7-505430812ae2\") " pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:51:39.536115 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.536087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9vwn\" (UniqueName: \"kubernetes.io/projected/5a733006-b68c-4968-b5f5-8fc583b72a47-kube-api-access-w9vwn\") pod \"authorino-58cd955675-vqlmz\" (UID: \"5a733006-b68c-4968-b5f5-8fc583b72a47\") " pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:39.544785 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.544750    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9vwn\" (UniqueName: \"kubernetes.io/projected/5a733006-b68c-4968-b5f5-8fc583b72a47-kube-api-access-w9vwn\") pod \"authorino-58cd955675-vqlmz\" (UID: \"5a733006-b68c-4968-b5f5-8fc583b72a47\") " pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:39.544950 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.544793    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrgc6\" (UniqueName: \"kubernetes.io/projected/9dfd43c1-1a94-4cd7-afc7-505430812ae2-kube-api-access-zrgc6\") pod \"authorino-75d556bd77-wlzzn\" (UID: \"9dfd43c1-1a94-4cd7-afc7-505430812ae2\") " pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:51:39.678515 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.678479    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:39.688626 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.688598    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:51:39.880118 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.880087    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vqlmz"]
Jul 08 02:51:39.882797 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:51:39.882765    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a733006_b68c_4968_b5f5_8fc583b72a47.slice/crio-23674e42d46fdae65e7c754a5d59d6212e97965c8c25140a51345a3340701bb3 WatchSource:0}: Error finding container 23674e42d46fdae65e7c754a5d59d6212e97965c8c25140a51345a3340701bb3: Status 404 returned error can't find the container with id 23674e42d46fdae65e7c754a5d59d6212e97965c8c25140a51345a3340701bb3
Jul 08 02:51:39.899999 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:39.899973    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlzzn"]
Jul 08 02:51:39.902575 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:51:39.902553    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dfd43c1_1a94_4cd7_afc7_505430812ae2.slice/crio-1ceffb0087331bcafde5fd8586eb38ea639f87b54e2a7734f5da849cece0b654 WatchSource:0}: Error finding container 1ceffb0087331bcafde5fd8586eb38ea639f87b54e2a7734f5da849cece0b654: Status 404 returned error can't find the container with id 1ceffb0087331bcafde5fd8586eb38ea639f87b54e2a7734f5da849cece0b654
Jul 08 02:51:40.196904 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:40.196872    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vqlmz" event={"ID":"5a733006-b68c-4968-b5f5-8fc583b72a47","Type":"ContainerStarted","Data":"23674e42d46fdae65e7c754a5d59d6212e97965c8c25140a51345a3340701bb3"}
Jul 08 02:51:40.198201 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:40.198171    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlzzn" event={"ID":"9dfd43c1-1a94-4cd7-afc7-505430812ae2","Type":"ContainerStarted","Data":"1ceffb0087331bcafde5fd8586eb38ea639f87b54e2a7734f5da849cece0b654"}
Jul 08 02:51:44.225116 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:44.225079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vqlmz" event={"ID":"5a733006-b68c-4968-b5f5-8fc583b72a47","Type":"ContainerStarted","Data":"9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4"}
Jul 08 02:51:44.226597 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:44.226576    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlzzn" event={"ID":"9dfd43c1-1a94-4cd7-afc7-505430812ae2","Type":"ContainerStarted","Data":"ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8"}
Jul 08 02:51:44.240896 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:44.240831    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vqlmz" podStartSLOduration=1.841073768 podStartE2EDuration="5.240818088s" totalImagesPullingTime="3.39974432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:51:44.239898333 +0000 UTC m=+692.167759944" watchObservedRunningTime="2026-07-08 02:51:44.240818088 +0000 UTC m=+692.168679737"
Jul 08 02:51:44.256251 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:44.256207    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wlzzn" podStartSLOduration=1.865950485 podStartE2EDuration="5.256194731s" totalImagesPullingTime="3.390244246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:51:44.254251822 +0000 UTC m=+692.182113431" watchObservedRunningTime="2026-07-08 02:51:44.256194731 +0000 UTC m=+692.184056340"
Jul 08 02:51:44.298097 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:44.298065    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vqlmz"]
Jul 08 02:51:46.235473 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:46.235437    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vqlmz" podUID="5a733006-b68c-4968-b5f5-8fc583b72a47" containerName="authorino" containerID="cri-o://9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4" gracePeriod=30
Jul 08 02:51:46.481085 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:46.481052    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:46.600426 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:46.600313    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9vwn\" (UniqueName: \"kubernetes.io/projected/5a733006-b68c-4968-b5f5-8fc583b72a47-kube-api-access-w9vwn\") pod \"5a733006-b68c-4968-b5f5-8fc583b72a47\" (UID: \"5a733006-b68c-4968-b5f5-8fc583b72a47\") "
Jul 08 02:51:46.602507 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:46.602476    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a733006-b68c-4968-b5f5-8fc583b72a47-kube-api-access-w9vwn" (OuterVolumeSpecName: "kube-api-access-w9vwn") pod "5a733006-b68c-4968-b5f5-8fc583b72a47" (UID: "5a733006-b68c-4968-b5f5-8fc583b72a47"). InnerVolumeSpecName "kube-api-access-w9vwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:51:46.701039 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:46.701009    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9vwn\" (UniqueName: \"kubernetes.io/projected/5a733006-b68c-4968-b5f5-8fc583b72a47-kube-api-access-w9vwn\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:51:47.240767 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.240727    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a733006-b68c-4968-b5f5-8fc583b72a47" containerID="9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4" exitCode=0
Jul 08 02:51:47.240767 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.240768    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vqlmz"
Jul 08 02:51:47.241243 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.240773    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vqlmz" event={"ID":"5a733006-b68c-4968-b5f5-8fc583b72a47","Type":"ContainerDied","Data":"9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4"}
Jul 08 02:51:47.241243 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.240828    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vqlmz" event={"ID":"5a733006-b68c-4968-b5f5-8fc583b72a47","Type":"ContainerDied","Data":"23674e42d46fdae65e7c754a5d59d6212e97965c8c25140a51345a3340701bb3"}
Jul 08 02:51:47.241243 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.240861    2558 scope.go:117] "RemoveContainer" containerID="9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4"
Jul 08 02:51:47.249283 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.249265    2558 scope.go:117] "RemoveContainer" containerID="9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4"
Jul 08 02:51:47.250015 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:51:47.249996    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4\": container with ID starting with 9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4 not found: ID does not exist" containerID="9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4"
Jul 08 02:51:47.250069 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.250025    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4"} err="failed to get container status \"9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4\": rpc error: code = NotFound desc = could not find container \"9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4\": container with ID starting with 9b5d4a4a13933cde72806ff87fd840ae5b8386dffc91a19ca25cbf93bdd13ce4 not found: ID does not exist"
Jul 08 02:51:47.259777 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.259752    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vqlmz"]
Jul 08 02:51:47.263551 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:47.263530    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vqlmz"]
Jul 08 02:51:48.618938 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:51:48.618906    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a733006-b68c-4968-b5f5-8fc583b72a47" path="/var/lib/kubelet/pods/5a733006-b68c-4968-b5f5-8fc583b72a47/volumes"
Jul 08 02:52:11.617314 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.617274    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xn6l4"]
Jul 08 02:52:11.617714 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.617531    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a733006-b68c-4968-b5f5-8fc583b72a47" containerName="authorino"
Jul 08 02:52:11.617714 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.617544    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a733006-b68c-4968-b5f5-8fc583b72a47" containerName="authorino"
Jul 08 02:52:11.617714 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.617620    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a733006-b68c-4968-b5f5-8fc583b72a47" containerName="authorino"
Jul 08 02:52:11.619511 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.619495    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:11.626232 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.626069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xn6l4"]
Jul 08 02:52:11.700556 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.700519    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2k48\" (UniqueName: \"kubernetes.io/projected/f1687a4d-5fe4-442e-b50e-e4198af903ab-kube-api-access-m2k48\") pod \"authorino-5d4b67c446-xn6l4\" (UID: \"f1687a4d-5fe4-442e-b50e-e4198af903ab\") " pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:11.801125 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.801083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2k48\" (UniqueName: \"kubernetes.io/projected/f1687a4d-5fe4-442e-b50e-e4198af903ab-kube-api-access-m2k48\") pod \"authorino-5d4b67c446-xn6l4\" (UID: \"f1687a4d-5fe4-442e-b50e-e4198af903ab\") " pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:11.810956 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.810926    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2k48\" (UniqueName: \"kubernetes.io/projected/f1687a4d-5fe4-442e-b50e-e4198af903ab-kube-api-access-m2k48\") pod \"authorino-5d4b67c446-xn6l4\" (UID: \"f1687a4d-5fe4-442e-b50e-e4198af903ab\") " pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:11.864992 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.864958    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xn6l4"]
Jul 08 02:52:11.865195 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.865183    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:11.901443 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.901408    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cbbd74845-8fvn8"]
Jul 08 02:52:11.904237 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.904212    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:11.913304 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.913243    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cbbd74845-8fvn8"]
Jul 08 02:52:11.958440 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.958404    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cbbd74845-8fvn8"]
Jul 08 02:52:11.958698 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:52:11.958674    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ct2p6], unattached volumes=[], failed to process volumes=[kube-api-access-ct2p6]: context canceled" pod="kuadrant-system/authorino-cbbd74845-8fvn8" podUID="e34f4f80-565b-42ba-86f3-3e1a7d3e32f2"
Jul 08 02:52:11.986995 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.986964    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6477784785-gkxgg"]
Jul 08 02:52:11.993242 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.993214    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:11.995831 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.995802    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 08 02:52:11.999522 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:11.999495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6477784785-gkxgg"]
Jul 08 02:52:12.002140 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.002119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct2p6\" (UniqueName: \"kubernetes.io/projected/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2-kube-api-access-ct2p6\") pod \"authorino-cbbd74845-8fvn8\" (UID: \"e34f4f80-565b-42ba-86f3-3e1a7d3e32f2\") " pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:12.006042 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.006019    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xn6l4"]
Jul 08 02:52:12.009178 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:52:12.009154    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1687a4d_5fe4_442e_b50e_e4198af903ab.slice/crio-4140d9e23dc610d855d6f501cee369aab7febf1acd7949476b4121ef03d0f68c WatchSource:0}: Error finding container 4140d9e23dc610d855d6f501cee369aab7febf1acd7949476b4121ef03d0f68c: Status 404 returned error can't find the container with id 4140d9e23dc610d855d6f501cee369aab7febf1acd7949476b4121ef03d0f68c
Jul 08 02:52:12.102733 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.102697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct2p6\" (UniqueName: \"kubernetes.io/projected/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2-kube-api-access-ct2p6\") pod \"authorino-cbbd74845-8fvn8\" (UID: \"e34f4f80-565b-42ba-86f3-3e1a7d3e32f2\") " pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:12.102733 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.102741    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-tls-cert\") pod \"authorino-6477784785-gkxgg\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") " pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.102967 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.102762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlcx4\" (UniqueName: \"kubernetes.io/projected/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-kube-api-access-zlcx4\") pod \"authorino-6477784785-gkxgg\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") " pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.110741 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.110717    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct2p6\" (UniqueName: \"kubernetes.io/projected/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2-kube-api-access-ct2p6\") pod \"authorino-cbbd74845-8fvn8\" (UID: \"e34f4f80-565b-42ba-86f3-3e1a7d3e32f2\") " pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:12.204157 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.204120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlcx4\" (UniqueName: \"kubernetes.io/projected/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-kube-api-access-zlcx4\") pod \"authorino-6477784785-gkxgg\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") " pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.204480 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.204225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-tls-cert\") pod \"authorino-6477784785-gkxgg\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") " pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.206725 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.206703    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-tls-cert\") pod \"authorino-6477784785-gkxgg\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") " pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.211782 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.211764    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlcx4\" (UniqueName: \"kubernetes.io/projected/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-kube-api-access-zlcx4\") pod \"authorino-6477784785-gkxgg\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") " pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.302476 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.302437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:52:12.357819 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.357462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xn6l4" event={"ID":"f1687a4d-5fe4-442e-b50e-e4198af903ab","Type":"ContainerStarted","Data":"4140d9e23dc610d855d6f501cee369aab7febf1acd7949476b4121ef03d0f68c"}
Jul 08 02:52:12.357819 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.357668    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:12.364603 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.364580    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:12.404850 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.404800    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct2p6\" (UniqueName: \"kubernetes.io/projected/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2-kube-api-access-ct2p6\") pod \"e34f4f80-565b-42ba-86f3-3e1a7d3e32f2\" (UID: \"e34f4f80-565b-42ba-86f3-3e1a7d3e32f2\") "
Jul 08 02:52:12.407165 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.407137    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2-kube-api-access-ct2p6" (OuterVolumeSpecName: "kube-api-access-ct2p6") pod "e34f4f80-565b-42ba-86f3-3e1a7d3e32f2" (UID: "e34f4f80-565b-42ba-86f3-3e1a7d3e32f2"). InnerVolumeSpecName "kube-api-access-ct2p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:52:12.432382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.432362    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6477784785-gkxgg"]
Jul 08 02:52:12.434667 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:52:12.434644    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5e13b75_01a1_46ed_a7b3_285b5f77a7be.slice/crio-ede68fef5ef90e63877fd38d573ce3b2eb71c2ca3282239b500d750081cd3016 WatchSource:0}: Error finding container ede68fef5ef90e63877fd38d573ce3b2eb71c2ca3282239b500d750081cd3016: Status 404 returned error can't find the container with id ede68fef5ef90e63877fd38d573ce3b2eb71c2ca3282239b500d750081cd3016
Jul 08 02:52:12.505548 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:12.505503    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct2p6\" (UniqueName: \"kubernetes.io/projected/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2-kube-api-access-ct2p6\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:52:13.363920 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.363871    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xn6l4" event={"ID":"f1687a4d-5fe4-442e-b50e-e4198af903ab","Type":"ContainerStarted","Data":"634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d"}
Jul 08 02:52:13.364382 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.363940    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-xn6l4" podUID="f1687a4d-5fe4-442e-b50e-e4198af903ab" containerName="authorino" containerID="cri-o://634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d" gracePeriod=30
Jul 08 02:52:13.365639 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.365611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6477784785-gkxgg" event={"ID":"c5e13b75-01a1-46ed-a7b3-285b5f77a7be","Type":"ContainerStarted","Data":"259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027"}
Jul 08 02:52:13.365639 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.365636    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbbd74845-8fvn8"
Jul 08 02:52:13.365796 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.365650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6477784785-gkxgg" event={"ID":"c5e13b75-01a1-46ed-a7b3-285b5f77a7be","Type":"ContainerStarted","Data":"ede68fef5ef90e63877fd38d573ce3b2eb71c2ca3282239b500d750081cd3016"}
Jul 08 02:52:13.381708 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.381667    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-xn6l4" podStartSLOduration=2.034847672 podStartE2EDuration="2.381649684s" totalImagesPullingTime="346.802012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:52:13.379818275 +0000 UTC m=+721.307679885" watchObservedRunningTime="2026-07-08 02:52:13.381649684 +0000 UTC m=+721.309511292"
Jul 08 02:52:13.404043 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.404014    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cbbd74845-8fvn8"]
Jul 08 02:52:13.406413 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.406385    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cbbd74845-8fvn8"]
Jul 08 02:52:13.423082 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.423042    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6477784785-gkxgg" podStartSLOduration=2.11160978 podStartE2EDuration="2.423026796s" totalImagesPullingTime="311.417016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:52:13.422207957 +0000 UTC m=+721.350069566" watchObservedRunningTime="2026-07-08 02:52:13.423026796 +0000 UTC m=+721.350888404"
Jul 08 02:52:13.448255 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.448223    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlzzn"]
Jul 08 02:52:13.448438 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.448387    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wlzzn" podUID="9dfd43c1-1a94-4cd7-afc7-505430812ae2" containerName="authorino" containerID="cri-o://ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8" gracePeriod=30
Jul 08 02:52:13.700109 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.700087    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:13.703009 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.702989    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:52:13.816823 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.816782    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrgc6\" (UniqueName: \"kubernetes.io/projected/9dfd43c1-1a94-4cd7-afc7-505430812ae2-kube-api-access-zrgc6\") pod \"9dfd43c1-1a94-4cd7-afc7-505430812ae2\" (UID: \"9dfd43c1-1a94-4cd7-afc7-505430812ae2\") "
Jul 08 02:52:13.817038 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.816832    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2k48\" (UniqueName: \"kubernetes.io/projected/f1687a4d-5fe4-442e-b50e-e4198af903ab-kube-api-access-m2k48\") pod \"f1687a4d-5fe4-442e-b50e-e4198af903ab\" (UID: \"f1687a4d-5fe4-442e-b50e-e4198af903ab\") "
Jul 08 02:52:13.819000 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.818968    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1687a4d-5fe4-442e-b50e-e4198af903ab-kube-api-access-m2k48" (OuterVolumeSpecName: "kube-api-access-m2k48") pod "f1687a4d-5fe4-442e-b50e-e4198af903ab" (UID: "f1687a4d-5fe4-442e-b50e-e4198af903ab"). InnerVolumeSpecName "kube-api-access-m2k48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:52:13.819000 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.818979    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dfd43c1-1a94-4cd7-afc7-505430812ae2-kube-api-access-zrgc6" (OuterVolumeSpecName: "kube-api-access-zrgc6") pod "9dfd43c1-1a94-4cd7-afc7-505430812ae2" (UID: "9dfd43c1-1a94-4cd7-afc7-505430812ae2"). InnerVolumeSpecName "kube-api-access-zrgc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:52:13.917467 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.917437    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2k48\" (UniqueName: \"kubernetes.io/projected/f1687a4d-5fe4-442e-b50e-e4198af903ab-kube-api-access-m2k48\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:52:13.917467 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:13.917465    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrgc6\" (UniqueName: \"kubernetes.io/projected/9dfd43c1-1a94-4cd7-afc7-505430812ae2-kube-api-access-zrgc6\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:52:14.371750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.371646    2558 generic.go:356] "Generic (PLEG): container finished" podID="9dfd43c1-1a94-4cd7-afc7-505430812ae2" containerID="ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8" exitCode=0
Jul 08 02:52:14.371750 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.371713    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlzzn"
Jul 08 02:52:14.372255 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.371741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlzzn" event={"ID":"9dfd43c1-1a94-4cd7-afc7-505430812ae2","Type":"ContainerDied","Data":"ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8"}
Jul 08 02:52:14.372255 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.371797    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlzzn" event={"ID":"9dfd43c1-1a94-4cd7-afc7-505430812ae2","Type":"ContainerDied","Data":"1ceffb0087331bcafde5fd8586eb38ea639f87b54e2a7734f5da849cece0b654"}
Jul 08 02:52:14.372255 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.371821    2558 scope.go:117] "RemoveContainer" containerID="ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8"
Jul 08 02:52:14.373156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.373065    2558 generic.go:356] "Generic (PLEG): container finished" podID="f1687a4d-5fe4-442e-b50e-e4198af903ab" containerID="634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d" exitCode=0
Jul 08 02:52:14.373156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.373104    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xn6l4" event={"ID":"f1687a4d-5fe4-442e-b50e-e4198af903ab","Type":"ContainerDied","Data":"634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d"}
Jul 08 02:52:14.373156 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.373138    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xn6l4"
Jul 08 02:52:14.373274 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.373142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xn6l4" event={"ID":"f1687a4d-5fe4-442e-b50e-e4198af903ab","Type":"ContainerDied","Data":"4140d9e23dc610d855d6f501cee369aab7febf1acd7949476b4121ef03d0f68c"}
Jul 08 02:52:14.383667 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.383576    2558 scope.go:117] "RemoveContainer" containerID="ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8"
Jul 08 02:52:14.383949 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:52:14.383909    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8\": container with ID starting with ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8 not found: ID does not exist" containerID="ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8"
Jul 08 02:52:14.384075 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.383946    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8"} err="failed to get container status \"ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8\": rpc error: code = NotFound desc = could not find container \"ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8\": container with ID starting with ded0432f3a64ab0813eeb334a542506f3f65e4d716e45cb58d503903039221b8 not found: ID does not exist"
Jul 08 02:52:14.384075 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.383969    2558 scope.go:117] "RemoveContainer" containerID="634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d"
Jul 08 02:52:14.392125 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.392105    2558 scope.go:117] "RemoveContainer" containerID="634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d"
Jul 08 02:52:14.392387 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:52:14.392369    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d\": container with ID starting with 634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d not found: ID does not exist" containerID="634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d"
Jul 08 02:52:14.392446 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.392400    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d"} err="failed to get container status \"634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d\": rpc error: code = NotFound desc = could not find container \"634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d\": container with ID starting with 634f896ccbffd8df1e8b709ab8a945a2f937836e9b016c7a37dd16dd9a591a4d not found: ID does not exist"
Jul 08 02:52:14.398570 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.398552    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xn6l4"]
Jul 08 02:52:14.402414 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.402393    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xn6l4"]
Jul 08 02:52:14.412047 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.412028    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlzzn"]
Jul 08 02:52:14.415983 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.415964    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlzzn"]
Jul 08 02:52:14.619064 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.619033    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dfd43c1-1a94-4cd7-afc7-505430812ae2" path="/var/lib/kubelet/pods/9dfd43c1-1a94-4cd7-afc7-505430812ae2/volumes"
Jul 08 02:52:14.619336 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.619325    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e34f4f80-565b-42ba-86f3-3e1a7d3e32f2" path="/var/lib/kubelet/pods/e34f4f80-565b-42ba-86f3-3e1a7d3e32f2/volumes"
Jul 08 02:52:14.619503 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:14.619492    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1687a4d-5fe4-442e-b50e-e4198af903ab" path="/var/lib/kubelet/pods/f1687a4d-5fe4-442e-b50e-e4198af903ab/volumes"
Jul 08 02:52:51.440142 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440105    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"]
Jul 08 02:52:51.440602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440378    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dfd43c1-1a94-4cd7-afc7-505430812ae2" containerName="authorino"
Jul 08 02:52:51.440602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440388    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dfd43c1-1a94-4cd7-afc7-505430812ae2" containerName="authorino"
Jul 08 02:52:51.440602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440410    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1687a4d-5fe4-442e-b50e-e4198af903ab" containerName="authorino"
Jul 08 02:52:51.440602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440415    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1687a4d-5fe4-442e-b50e-e4198af903ab" containerName="authorino"
Jul 08 02:52:51.440602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440455    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dfd43c1-1a94-4cd7-afc7-505430812ae2" containerName="authorino"
Jul 08 02:52:51.440602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.440463    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1687a4d-5fe4-442e-b50e-e4198af903ab" containerName="authorino"
Jul 08 02:52:51.443271 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.443255    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.445872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.445829    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 08 02:52:51.445998 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.445873    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 08 02:52:51.446786 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.446766    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 08 02:52:51.446887 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.446793    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h8t2l\""
Jul 08 02:52:51.454393 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.454367    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"]
Jul 08 02:52:51.528323 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.528284    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-maas-api-tls\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.528519 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.528387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-openshift-service-ca\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.528519 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.528418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4495\" (UniqueName: \"kubernetes.io/projected/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-kube-api-access-c4495\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.629114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.629077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-openshift-service-ca\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.629114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.629113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4495\" (UniqueName: \"kubernetes.io/projected/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-kube-api-access-c4495\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.629364 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.629186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-maas-api-tls\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.629364 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:52:51.629299    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 08 02:52:51.629464 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:52:51.629390    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-maas-api-tls podName:25a9d562-a08e-47d0-aadb-fbac3b06a6e6 nodeName:}" failed. No retries permitted until 2026-07-08 02:52:52.129367194 +0000 UTC m=+760.057228782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-maas-api-tls") pod "maas-api-8577dfc8d7-xfljf" (UID: "25a9d562-a08e-47d0-aadb-fbac3b06a6e6") : secret "maas-api-serving-cert" not found
Jul 08 02:52:51.629800 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.629781    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-openshift-service-ca\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:51.639832 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:51.639801    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4495\" (UniqueName: \"kubernetes.io/projected/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-kube-api-access-c4495\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:52.133964 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:52.133920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-maas-api-tls\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:52.136390 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:52.136366    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a9d562-a08e-47d0-aadb-fbac3b06a6e6-maas-api-tls\") pod \"maas-api-8577dfc8d7-xfljf\" (UID: \"25a9d562-a08e-47d0-aadb-fbac3b06a6e6\") " pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:52.353630 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:52.353597    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:52.487106 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:52.487081    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"]
Jul 08 02:52:52.489411 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:52:52.489385    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a9d562_a08e_47d0_aadb_fbac3b06a6e6.slice/crio-9b1afd17ccaeff4069f2b5e09793ff65073eba40c1c0921eb450199e704939ff WatchSource:0}: Error finding container 9b1afd17ccaeff4069f2b5e09793ff65073eba40c1c0921eb450199e704939ff: Status 404 returned error can't find the container with id 9b1afd17ccaeff4069f2b5e09793ff65073eba40c1c0921eb450199e704939ff
Jul 08 02:52:52.490875 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:52.490823    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 02:52:52.545598 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:52.545563    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf" event={"ID":"25a9d562-a08e-47d0-aadb-fbac3b06a6e6","Type":"ContainerStarted","Data":"9b1afd17ccaeff4069f2b5e09793ff65073eba40c1c0921eb450199e704939ff"}
Jul 08 02:52:54.556963 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:54.556925    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf" event={"ID":"25a9d562-a08e-47d0-aadb-fbac3b06a6e6","Type":"ContainerStarted","Data":"2040bbc721c45f79f040e41ba040026db73ff3ed1d8bcebb373de15783a51a09"}
Jul 08 02:52:54.557414 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:54.557000    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:52:54.583562 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:52:54.583505    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf" podStartSLOduration=1.707942703 podStartE2EDuration="3.583485164s" totalImagesPullingTime="1.875542461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:52:54.583405448 +0000 UTC m=+762.511267061" watchObservedRunningTime="2026-07-08 02:52:54.583485164 +0000 UTC m=+762.511346772"
Jul 08 02:53:00.567430 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:00.567394    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8577dfc8d7-xfljf"
Jul 08 02:53:07.468135 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.468103    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"]
Jul 08 02:53:07.471207 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.471189    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.474716 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.474695    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4sql4\""
Jul 08 02:53:07.474872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.474744    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 08 02:53:07.474872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.474787    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 08 02:53:07.474990 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.474888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 08 02:53:07.484913 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.484891    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"]
Jul 08 02:53:07.565707 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.565675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.565707 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.565710    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.565997 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.565740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2vxz\" (UniqueName: \"kubernetes.io/projected/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-kube-api-access-q2vxz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.565997 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.565816    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.565997 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.565888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.565997 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.565923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667039 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667008    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667039 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667309 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667071    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2vxz\" (UniqueName: \"kubernetes.io/projected/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-kube-api-access-q2vxz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667309 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667309 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667123    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667309 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667602 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667577    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667686 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667586    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.667822 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.667799    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.669580 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.669507    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.669831 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.669812    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.677512 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.677491    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2vxz\" (UniqueName: \"kubernetes.io/projected/8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea-kube-api-access-q2vxz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2\" (UID: \"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.781661 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.781572    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:07.916384 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:07.916357    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"]
Jul 08 02:53:07.918283 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:53:07.918256    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a4ad3c3_f51b_4b67_b4a1_0d4d7ae0deea.slice/crio-feb4840c6782d75ae0226f082b5fd5edb927afdc390ba91adbb09b8454b530bd WatchSource:0}: Error finding container feb4840c6782d75ae0226f082b5fd5edb927afdc390ba91adbb09b8454b530bd: Status 404 returned error can't find the container with id feb4840c6782d75ae0226f082b5fd5edb927afdc390ba91adbb09b8454b530bd
Jul 08 02:53:08.623974 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:08.623933    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2" event={"ID":"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea","Type":"ContainerStarted","Data":"feb4840c6782d75ae0226f082b5fd5edb927afdc390ba91adbb09b8454b530bd"}
Jul 08 02:53:09.632616 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:09.632578    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2" event={"ID":"8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea","Type":"ContainerStarted","Data":"6030e1d3349c7882b795801f9774e2bae81aab87b1f83c179fcec6a5e9e132f6"}
Jul 08 02:53:09.654050 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:09.653997    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2" podStartSLOduration=1.613377108 podStartE2EDuration="2.653982939s" totalImagesPullingTime="1.040605831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:53:09.652015506 +0000 UTC m=+777.579877116" watchObservedRunningTime="2026-07-08 02:53:09.653982939 +0000 UTC m=+777.581844550"
Jul 08 02:53:17.781724 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:17.781682    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:17.782179 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:17.781782    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:17.794085 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:17.794064    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:53:18.682695 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:53:18.682667    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2"
Jul 08 02:54:10.384803 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.384768    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"]
Jul 08 02:54:10.387911 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.387894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.391892 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.391873    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 08 02:54:10.418059 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.418029    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"]
Jul 08 02:54:10.495635 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.495596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.495635 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.495636    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.495872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.495658    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnfw\" (UniqueName: \"kubernetes.io/projected/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-kube-api-access-5lnfw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.495872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.495714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.495872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.495781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.495872 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.495804    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597246 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597246 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnfw\" (UniqueName: \"kubernetes.io/projected/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-kube-api-access-5lnfw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597289    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597322    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597719 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597696    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597761 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597708    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.597796 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.597770    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.599540 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.599521    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.599957 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.599939    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.608833 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.608806    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnfw\" (UniqueName: \"kubernetes.io/projected/7e2b5d2e-bc56-4578-9a7a-3d01411cb708-kube-api-access-5lnfw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m97xr\" (UID: \"7e2b5d2e-bc56-4578-9a7a-3d01411cb708\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.698166 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.698124    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:10.837407 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.837383    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"]
Jul 08 02:54:10.839670 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:54:10.839646    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2b5d2e_bc56_4578_9a7a_3d01411cb708.slice/crio-a3b0b17b89a9d681adee61bfb0e7d99364ad5d91adace5c699d4a08f2265922b WatchSource:0}: Error finding container a3b0b17b89a9d681adee61bfb0e7d99364ad5d91adace5c699d4a08f2265922b: Status 404 returned error can't find the container with id a3b0b17b89a9d681adee61bfb0e7d99364ad5d91adace5c699d4a08f2265922b
Jul 08 02:54:10.898759 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:10.898718    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr" event={"ID":"7e2b5d2e-bc56-4578-9a7a-3d01411cb708","Type":"ContainerStarted","Data":"a3b0b17b89a9d681adee61bfb0e7d99364ad5d91adace5c699d4a08f2265922b"}
Jul 08 02:54:11.904704 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:11.904663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr" event={"ID":"7e2b5d2e-bc56-4578-9a7a-3d01411cb708","Type":"ContainerStarted","Data":"f61ce091c24d68df5af9ff2e428a349f5cb886432107057f4ca7e10bae5d90f2"}
Jul 08 02:54:11.924726 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:11.924672    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr" podStartSLOduration=1.740593591 podStartE2EDuration="1.924655407s" totalImagesPullingTime="184.061816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:54:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:54:11.92423729 +0000 UTC m=+839.852098903" watchObservedRunningTime="2026-07-08 02:54:11.924655407 +0000 UTC m=+839.852517016"
Jul 08 02:54:20.699113 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:20.699072    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:20.699585 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:20.699146    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:20.711390 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:20.711361    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:20.957517 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:20.957434    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m97xr"
Jul 08 02:54:46.758966 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.758929    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"]
Jul 08 02:54:46.762065 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.762048    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.764637 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.764612    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 08 02:54:46.773022 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.772997    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"]
Jul 08 02:54:46.818734 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.818685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dch8\" (UniqueName: \"kubernetes.io/projected/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-kube-api-access-7dch8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.818734 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.818726    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.819058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.818774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.819058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.818790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.819058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.818875    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.819058 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.818910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920155 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920155 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920419 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920419 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920419 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dch8\" (UniqueName: \"kubernetes.io/projected/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-kube-api-access-7dch8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920419 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920653 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920631    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920719 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920704    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.920769 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.920709    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.923092 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.923061    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.923440 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.923418    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:46.928185 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:46.928166    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dch8\" (UniqueName: \"kubernetes.io/projected/f2b6d27e-fe48-418a-a8b8-6ae59aa8af63-kube-api-access-7dch8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2k6bz\" (UID: \"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:47.072482 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:47.072395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:47.206079 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:47.206055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"]
Jul 08 02:54:47.208135 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:54:47.208108    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b6d27e_fe48_418a_a8b8_6ae59aa8af63.slice/crio-fcc5959cdd58e1bd6d7e1afbaba368ce80e69aa94c869115022966161e2e8bf0 WatchSource:0}: Error finding container fcc5959cdd58e1bd6d7e1afbaba368ce80e69aa94c869115022966161e2e8bf0: Status 404 returned error can't find the container with id fcc5959cdd58e1bd6d7e1afbaba368ce80e69aa94c869115022966161e2e8bf0
Jul 08 02:54:48.071085 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:48.071045    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz" event={"ID":"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63","Type":"ContainerStarted","Data":"a757b5f58d18c4e1af87574610ffb4e2df96195554a9e06b4aa80f983943e701"}
Jul 08 02:54:48.071488 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:48.071114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz" event={"ID":"f2b6d27e-fe48-418a-a8b8-6ae59aa8af63","Type":"ContainerStarted","Data":"fcc5959cdd58e1bd6d7e1afbaba368ce80e69aa94c869115022966161e2e8bf0"}
Jul 08 02:54:48.089343 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:48.089287    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz" podStartSLOduration=1.920527357 podStartE2EDuration="2.089268925s" totalImagesPullingTime="168.741568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:54:48.087875214 +0000 UTC m=+876.015736819" watchObservedRunningTime="2026-07-08 02:54:48.089268925 +0000 UTC m=+876.017130534"
Jul 08 02:54:49.037157 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.037122    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"]
Jul 08 02:54:49.040779 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.040758    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.043184 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.043159    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 08 02:54:49.051148 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.051121    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"]
Jul 08 02:54:49.138421 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.138380    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ebbeaf-8cc2-4477-b438-4f51621d69c0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.138908 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.138429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfp2g\" (UniqueName: \"kubernetes.io/projected/83ebbeaf-8cc2-4477-b438-4f51621d69c0-kube-api-access-jfp2g\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.138908 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.138502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.138908 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.138578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.138908 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.138596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.138908 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.138653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240189 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240391 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240391 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240247    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240391 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240391 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ebbeaf-8cc2-4477-b438-4f51621d69c0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240391 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240370    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfp2g\" (UniqueName: \"kubernetes.io/projected/83ebbeaf-8cc2-4477-b438-4f51621d69c0-kube-api-access-jfp2g\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240695 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240664    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240757 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240695    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.240813 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.240753    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.242633 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.242614    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ebbeaf-8cc2-4477-b438-4f51621d69c0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.243019 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.242999    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ebbeaf-8cc2-4477-b438-4f51621d69c0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.256114 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.256080    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfp2g\" (UniqueName: \"kubernetes.io/projected/83ebbeaf-8cc2-4477-b438-4f51621d69c0-kube-api-access-jfp2g\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt\" (UID: \"83ebbeaf-8cc2-4477-b438-4f51621d69c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.351476 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.351375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:49.492963 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:49.492932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"]
Jul 08 02:54:49.494600 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:54:49.494573    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83ebbeaf_8cc2_4477_b438_4f51621d69c0.slice/crio-a293a6c0edf6f291bc1ca9194f76f63f5014180ae087eba25d3dee6a84158d2c WatchSource:0}: Error finding container a293a6c0edf6f291bc1ca9194f76f63f5014180ae087eba25d3dee6a84158d2c: Status 404 returned error can't find the container with id a293a6c0edf6f291bc1ca9194f76f63f5014180ae087eba25d3dee6a84158d2c
Jul 08 02:54:50.081335 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:50.081244    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt" event={"ID":"83ebbeaf-8cc2-4477-b438-4f51621d69c0","Type":"ContainerStarted","Data":"ccc0329b992f2d09da81c03a14bf1a6277353b21458b4ec2a7242e92fe118669"}
Jul 08 02:54:50.081335 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:50.081281    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt" event={"ID":"83ebbeaf-8cc2-4477-b438-4f51621d69c0","Type":"ContainerStarted","Data":"a293a6c0edf6f291bc1ca9194f76f63f5014180ae087eba25d3dee6a84158d2c"}
Jul 08 02:54:50.100155 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:50.100110    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt" podStartSLOduration=0.929983981 podStartE2EDuration="1.100095638s" totalImagesPullingTime="170.111657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:54:50.099294004 +0000 UTC m=+878.027155615" watchObservedRunningTime="2026-07-08 02:54:50.100095638 +0000 UTC m=+878.027957245"
Jul 08 02:54:57.073421 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:57.073384    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:57.074052 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:57.073452    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:57.085935 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:57.085904    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:57.129674 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:57.129643    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2k6bz"
Jul 08 02:54:59.352424 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:59.352384    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:59.352424 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:59.352435    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:54:59.364508 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:54:59.364474    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:55:00.142370 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:00.142340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt"
Jul 08 02:55:12.517477 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:12.517450    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:55:12.518192 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:12.518173    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 02:55:45.014389 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.014354    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dd5b94576-lxxn6"]
Jul 08 02:55:45.017551 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.017536    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.025805 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.025778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dd5b94576-lxxn6"]
Jul 08 02:55:45.104191 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.104149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmc8g\" (UniqueName: \"kubernetes.io/projected/2ae60fe1-3884-47f6-907a-f9f550efdf93-kube-api-access-jmc8g\") pod \"authorino-7dd5b94576-lxxn6\" (UID: \"2ae60fe1-3884-47f6-907a-f9f550efdf93\") " pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.104191 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.104201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2ae60fe1-3884-47f6-907a-f9f550efdf93-tls-cert\") pod \"authorino-7dd5b94576-lxxn6\" (UID: \"2ae60fe1-3884-47f6-907a-f9f550efdf93\") " pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.205751 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.205711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmc8g\" (UniqueName: \"kubernetes.io/projected/2ae60fe1-3884-47f6-907a-f9f550efdf93-kube-api-access-jmc8g\") pod \"authorino-7dd5b94576-lxxn6\" (UID: \"2ae60fe1-3884-47f6-907a-f9f550efdf93\") " pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.205948 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.205766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2ae60fe1-3884-47f6-907a-f9f550efdf93-tls-cert\") pod \"authorino-7dd5b94576-lxxn6\" (UID: \"2ae60fe1-3884-47f6-907a-f9f550efdf93\") " pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.208424 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.208389    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2ae60fe1-3884-47f6-907a-f9f550efdf93-tls-cert\") pod \"authorino-7dd5b94576-lxxn6\" (UID: \"2ae60fe1-3884-47f6-907a-f9f550efdf93\") " pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.214563 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.214532    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmc8g\" (UniqueName: \"kubernetes.io/projected/2ae60fe1-3884-47f6-907a-f9f550efdf93-kube-api-access-jmc8g\") pod \"authorino-7dd5b94576-lxxn6\" (UID: \"2ae60fe1-3884-47f6-907a-f9f550efdf93\") " pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.327165 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.327069    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dd5b94576-lxxn6"
Jul 08 02:55:45.461386 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:45.461361    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dd5b94576-lxxn6"]
Jul 08 02:55:45.463426 ip-10-0-129-88 kubenswrapper[2558]: W0708 02:55:45.463406    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ae60fe1_3884_47f6_907a_f9f550efdf93.slice/crio-d3b16442db4d8ec7775fcb9630a4e1c5d2fb9ad68f0bcfc5b9ff2a7c14d33e35 WatchSource:0}: Error finding container d3b16442db4d8ec7775fcb9630a4e1c5d2fb9ad68f0bcfc5b9ff2a7c14d33e35: Status 404 returned error can't find the container with id d3b16442db4d8ec7775fcb9630a4e1c5d2fb9ad68f0bcfc5b9ff2a7c14d33e35
Jul 08 02:55:46.356777 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.356743    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dd5b94576-lxxn6" event={"ID":"2ae60fe1-3884-47f6-907a-f9f550efdf93","Type":"ContainerStarted","Data":"f33b51eb8c96653331f80da89012e6b0b83bbdb604ac489650830c84dfc0ea88"}
Jul 08 02:55:46.357261 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.356794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dd5b94576-lxxn6" event={"ID":"2ae60fe1-3884-47f6-907a-f9f550efdf93","Type":"ContainerStarted","Data":"d3b16442db4d8ec7775fcb9630a4e1c5d2fb9ad68f0bcfc5b9ff2a7c14d33e35"}
Jul 08 02:55:46.372889 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.372813    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7dd5b94576-lxxn6" podStartSLOduration=1.967367221 podStartE2EDuration="2.372795355s" totalImagesPullingTime="405.428134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:55:46.372389136 +0000 UTC m=+934.300250747" watchObservedRunningTime="2026-07-08 02:55:46.372795355 +0000 UTC m=+934.300656964"
Jul 08 02:55:46.407389 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.407310    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6477784785-gkxgg"]
Jul 08 02:55:46.407581 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.407513    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6477784785-gkxgg" podUID="c5e13b75-01a1-46ed-a7b3-285b5f77a7be" containerName="authorino" containerID="cri-o://259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027" gracePeriod=30
Jul 08 02:55:46.678380 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.678348    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:55:46.719275 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.719230    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlcx4\" (UniqueName: \"kubernetes.io/projected/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-kube-api-access-zlcx4\") pod \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") "
Jul 08 02:55:46.719458 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.719357    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-tls-cert\") pod \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\" (UID: \"c5e13b75-01a1-46ed-a7b3-285b5f77a7be\") "
Jul 08 02:55:46.721658 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.721615    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-kube-api-access-zlcx4" (OuterVolumeSpecName: "kube-api-access-zlcx4") pod "c5e13b75-01a1-46ed-a7b3-285b5f77a7be" (UID: "c5e13b75-01a1-46ed-a7b3-285b5f77a7be"). InnerVolumeSpecName "kube-api-access-zlcx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 02:55:46.730388 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.730356    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c5e13b75-01a1-46ed-a7b3-285b5f77a7be" (UID: "c5e13b75-01a1-46ed-a7b3-285b5f77a7be"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 02:55:46.821056 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.821016    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlcx4\" (UniqueName: \"kubernetes.io/projected/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-kube-api-access-zlcx4\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:55:46.821056 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:46.821052    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5e13b75-01a1-46ed-a7b3-285b5f77a7be-tls-cert\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 02:55:47.363504 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.363442    2558 generic.go:356] "Generic (PLEG): container finished" podID="c5e13b75-01a1-46ed-a7b3-285b5f77a7be" containerID="259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027" exitCode=0
Jul 08 02:55:47.364225 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.364193    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6477784785-gkxgg" event={"ID":"c5e13b75-01a1-46ed-a7b3-285b5f77a7be","Type":"ContainerDied","Data":"259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027"}
Jul 08 02:55:47.364301 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.364267    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6477784785-gkxgg" event={"ID":"c5e13b75-01a1-46ed-a7b3-285b5f77a7be","Type":"ContainerDied","Data":"ede68fef5ef90e63877fd38d573ce3b2eb71c2ca3282239b500d750081cd3016"}
Jul 08 02:55:47.364301 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.364291    2558 scope.go:117] "RemoveContainer" containerID="259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027"
Jul 08 02:55:47.364421 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.364404    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6477784785-gkxgg"
Jul 08 02:55:47.389081 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.389052    2558 scope.go:117] "RemoveContainer" containerID="259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027"
Jul 08 02:55:47.389607 ip-10-0-129-88 kubenswrapper[2558]: E0708 02:55:47.389584    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027\": container with ID starting with 259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027 not found: ID does not exist" containerID="259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027"
Jul 08 02:55:47.389744 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.389722    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027"} err="failed to get container status \"259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027\": rpc error: code = NotFound desc = could not find container \"259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027\": container with ID starting with 259e65fba5cbcbc2135f44f1e7875b9745257e76ae0a38518f3c582342fa4027 not found: ID does not exist"
Jul 08 02:55:47.403768 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.403732    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6477784785-gkxgg"]
Jul 08 02:55:47.412705 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:47.412677    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6477784785-gkxgg"]
Jul 08 02:55:48.619001 ip-10-0-129-88 kubenswrapper[2558]: I0708 02:55:48.618960    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e13b75-01a1-46ed-a7b3-285b5f77a7be" path="/var/lib/kubelet/pods/c5e13b75-01a1-46ed-a7b3-285b5f77a7be/volumes"
Jul 08 03:00:12.551430 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:00:12.551396    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:00:12.555739 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:00:12.555718    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:05:12.581331 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:05:12.581301    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:05:12.587923 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:05:12.587903    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:09:18.405951 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.405908    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"]
Jul 08 03:09:18.406330 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.406204    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e13b75-01a1-46ed-a7b3-285b5f77a7be" containerName="authorino"
Jul 08 03:09:18.406330 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.406216    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e13b75-01a1-46ed-a7b3-285b5f77a7be" containerName="authorino"
Jul 08 03:09:18.406330 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.406266    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e13b75-01a1-46ed-a7b3-285b5f77a7be" containerName="authorino"
Jul 08 03:09:18.409157 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.409141    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.412286 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.412251    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-15fb6db9-serving-cert\""
Jul 08 03:09:18.423223 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.423195    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"]
Jul 08 03:09:18.521144 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.521119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cg6h\" (UniqueName: \"kubernetes.io/projected/89810625-ccfd-43e9-9d63-b262e1bb26f3-kube-api-access-5cg6h\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.521305 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.521160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.521305 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.521179    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89810625-ccfd-43e9-9d63-b262e1bb26f3-openshift-service-ca\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.622141 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.622107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cg6h\" (UniqueName: \"kubernetes.io/projected/89810625-ccfd-43e9-9d63-b262e1bb26f3-kube-api-access-5cg6h\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.622294 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.622159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.622294 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.622180    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89810625-ccfd-43e9-9d63-b262e1bb26f3-openshift-service-ca\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.622294 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:18.622263    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-serving-cert: secret "maas-api-e2e-ait-15fb6db9-serving-cert" not found
Jul 08 03:09:18.622458 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:18.622339    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls podName:89810625-ccfd-43e9-9d63-b262e1bb26f3 nodeName:}" failed. No retries permitted until 2026-07-08 03:09:19.122318875 +0000 UTC m=+1747.050180462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls") pod "maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" (UID: "89810625-ccfd-43e9-9d63-b262e1bb26f3") : secret "maas-api-e2e-ait-15fb6db9-serving-cert" not found
Jul 08 03:09:18.622808 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.622786    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89810625-ccfd-43e9-9d63-b262e1bb26f3-openshift-service-ca\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:18.631343 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:18.631321    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cg6h\" (UniqueName: \"kubernetes.io/projected/89810625-ccfd-43e9-9d63-b262e1bb26f3-kube-api-access-5cg6h\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:19.127099 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:19.127064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:19.129484 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:19.129455    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls\") pod \"maas-api-e2e-ait-15fb6db9-f84df57bf-4v677\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:19.319154 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:19.319119    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:19.455806 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:19.455779    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"]
Jul 08 03:09:19.457425 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:09:19.457397    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89810625_ccfd_43e9_9d63_b262e1bb26f3.slice/crio-783fd0253bd8c0600ae93f174b405a5b09f3a52d480df2b93e82868b86c10702 WatchSource:0}: Error finding container 783fd0253bd8c0600ae93f174b405a5b09f3a52d480df2b93e82868b86c10702: Status 404 returned error can't find the container with id 783fd0253bd8c0600ae93f174b405a5b09f3a52d480df2b93e82868b86c10702
Jul 08 03:09:19.459092 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:19.459076    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:09:20.236203 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:20.236169    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" event={"ID":"89810625-ccfd-43e9-9d63-b262e1bb26f3","Type":"ContainerStarted","Data":"34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804"}
Jul 08 03:09:20.236373 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:20.236222    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" event={"ID":"89810625-ccfd-43e9-9d63-b262e1bb26f3","Type":"ContainerStarted","Data":"783fd0253bd8c0600ae93f174b405a5b09f3a52d480df2b93e82868b86c10702"}
Jul 08 03:09:20.236373 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:20.236256    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:20.256940 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:20.256891    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" podStartSLOduration=1.682019927 podStartE2EDuration="2.25687478s" totalImagesPullingTime="574.854853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:09:20.25608915 +0000 UTC m=+1748.183950761" watchObservedRunningTime="2026-07-08 03:09:20.25687478 +0000 UTC m=+1748.184736389"
Jul 08 03:09:26.247025 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.246995    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:26.325125 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.325090    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"]
Jul 08 03:09:26.325337 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.325306    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" podUID="89810625-ccfd-43e9-9d63-b262e1bb26f3" containerName="maas-api" containerID="cri-o://34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804" gracePeriod=30
Jul 08 03:09:26.385976 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:26.385951    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-serving-cert: secret "maas-api-e2e-ait-15fb6db9-serving-cert" not found
Jul 08 03:09:26.386094 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:26.386024    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls podName:89810625-ccfd-43e9-9d63-b262e1bb26f3 nodeName:}" failed. No retries permitted until 2026-07-08 03:09:26.886007775 +0000 UTC m=+1754.813869362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls") pod "maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" (UID: "89810625-ccfd-43e9-9d63-b262e1bb26f3") : secret "maas-api-e2e-ait-15fb6db9-serving-cert" not found
Jul 08 03:09:26.564414 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.564391    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:26.687630 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.687601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls\") pod \"89810625-ccfd-43e9-9d63-b262e1bb26f3\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") "
Jul 08 03:09:26.687806 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.687648    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cg6h\" (UniqueName: \"kubernetes.io/projected/89810625-ccfd-43e9-9d63-b262e1bb26f3-kube-api-access-5cg6h\") pod \"89810625-ccfd-43e9-9d63-b262e1bb26f3\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") "
Jul 08 03:09:26.687806 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.687665    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89810625-ccfd-43e9-9d63-b262e1bb26f3-openshift-service-ca\") pod \"89810625-ccfd-43e9-9d63-b262e1bb26f3\" (UID: \"89810625-ccfd-43e9-9d63-b262e1bb26f3\") "
Jul 08 03:09:26.688110 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.688083    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89810625-ccfd-43e9-9d63-b262e1bb26f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89810625-ccfd-43e9-9d63-b262e1bb26f3" (UID: "89810625-ccfd-43e9-9d63-b262e1bb26f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:09:26.690022 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.689998    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89810625-ccfd-43e9-9d63-b262e1bb26f3" (UID: "89810625-ccfd-43e9-9d63-b262e1bb26f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:09:26.690022 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.690017    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89810625-ccfd-43e9-9d63-b262e1bb26f3-kube-api-access-5cg6h" (OuterVolumeSpecName: "kube-api-access-5cg6h") pod "89810625-ccfd-43e9-9d63-b262e1bb26f3" (UID: "89810625-ccfd-43e9-9d63-b262e1bb26f3"). InnerVolumeSpecName "kube-api-access-5cg6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:09:26.789227 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.789200    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cg6h\" (UniqueName: \"kubernetes.io/projected/89810625-ccfd-43e9-9d63-b262e1bb26f3-kube-api-access-5cg6h\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:26.789227 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.789225    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89810625-ccfd-43e9-9d63-b262e1bb26f3-openshift-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:26.789374 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:26.789236    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89810625-ccfd-43e9-9d63-b262e1bb26f3-maas-api-tls\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:27.268055 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.268021    2558 generic.go:356] "Generic (PLEG): container finished" podID="89810625-ccfd-43e9-9d63-b262e1bb26f3" containerID="34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804" exitCode=0
Jul 08 03:09:27.268420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.268072    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"
Jul 08 03:09:27.268420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.268108    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" event={"ID":"89810625-ccfd-43e9-9d63-b262e1bb26f3","Type":"ContainerDied","Data":"34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804"}
Jul 08 03:09:27.268420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.268159    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677" event={"ID":"89810625-ccfd-43e9-9d63-b262e1bb26f3","Type":"ContainerDied","Data":"783fd0253bd8c0600ae93f174b405a5b09f3a52d480df2b93e82868b86c10702"}
Jul 08 03:09:27.268420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.268182    2558 scope.go:117] "RemoveContainer" containerID="34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804"
Jul 08 03:09:27.276867 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.276833    2558 scope.go:117] "RemoveContainer" containerID="34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804"
Jul 08 03:09:27.277115 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:27.277096    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804\": container with ID starting with 34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804 not found: ID does not exist" containerID="34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804"
Jul 08 03:09:27.277177 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.277122    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804"} err="failed to get container status \"34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804\": rpc error: code = NotFound desc = could not find container \"34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804\": container with ID starting with 34707e908d2314bdb4afe30803e88011a73dbba186bd3586556d0590b46ac804 not found: ID does not exist"
Jul 08 03:09:27.288862 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.288829    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"]
Jul 08 03:09:27.293134 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:27.293111    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-15fb6db9-f84df57bf-4v677"]
Jul 08 03:09:28.619527 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:28.619490    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89810625-ccfd-43e9-9d63-b262e1bb26f3" path="/var/lib/kubelet/pods/89810625-ccfd-43e9-9d63-b262e1bb26f3/volumes"
Jul 08 03:09:34.016403 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.016367    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"]
Jul 08 03:09:34.016878 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.016667    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89810625-ccfd-43e9-9d63-b262e1bb26f3" containerName="maas-api"
Jul 08 03:09:34.016878 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.016682    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="89810625-ccfd-43e9-9d63-b262e1bb26f3" containerName="maas-api"
Jul 08 03:09:34.016878 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.016751    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="89810625-ccfd-43e9-9d63-b262e1bb26f3" containerName="maas-api"
Jul 08 03:09:34.020967 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.020945    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.023432 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.023409    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-32f16294-serving-cert\""
Jul 08 03:09:34.041333 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.041304    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"]
Jul 08 03:09:34.152795 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.152758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef4e6ff6-8fdf-4019-bf08-be969178bec9-openshift-service-ca\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.152987 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.152820    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6p6d\" (UniqueName: \"kubernetes.io/projected/ef4e6ff6-8fdf-4019-bf08-be969178bec9-kube-api-access-x6p6d\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.152987 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.152873    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef4e6ff6-8fdf-4019-bf08-be969178bec9-maas-api-tls\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.253979 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.253935    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6p6d\" (UniqueName: \"kubernetes.io/projected/ef4e6ff6-8fdf-4019-bf08-be969178bec9-kube-api-access-x6p6d\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.254164 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.253992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef4e6ff6-8fdf-4019-bf08-be969178bec9-maas-api-tls\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.254164 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.254028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef4e6ff6-8fdf-4019-bf08-be969178bec9-openshift-service-ca\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.254605 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.254584    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef4e6ff6-8fdf-4019-bf08-be969178bec9-openshift-service-ca\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.256587 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.256561    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef4e6ff6-8fdf-4019-bf08-be969178bec9-maas-api-tls\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.263570 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.263549    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6p6d\" (UniqueName: \"kubernetes.io/projected/ef4e6ff6-8fdf-4019-bf08-be969178bec9-kube-api-access-x6p6d\") pod \"maas-api-e2e-ait-32f16294-69677f6f-4nmqf\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.331705 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.331638    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:34.473281 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:34.473257    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"]
Jul 08 03:09:34.474754 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:09:34.474727    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4e6ff6_8fdf_4019_bf08_be969178bec9.slice/crio-40ff63d96678396097ff124a19333ae1788169557555a46b059f9850574e7fc4 WatchSource:0}: Error finding container 40ff63d96678396097ff124a19333ae1788169557555a46b059f9850574e7fc4: Status 404 returned error can't find the container with id 40ff63d96678396097ff124a19333ae1788169557555a46b059f9850574e7fc4
Jul 08 03:09:35.306124 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:35.306077    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf" event={"ID":"ef4e6ff6-8fdf-4019-bf08-be969178bec9","Type":"ContainerStarted","Data":"83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913"}
Jul 08 03:09:35.306479 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:35.306146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf" event={"ID":"ef4e6ff6-8fdf-4019-bf08-be969178bec9","Type":"ContainerStarted","Data":"40ff63d96678396097ff124a19333ae1788169557555a46b059f9850574e7fc4"}
Jul 08 03:09:35.306479 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:35.306180    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:35.325125 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:35.325076    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf" podStartSLOduration=1.957333108 podStartE2EDuration="2.325062339s" totalImagesPullingTime="367.729231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:09:35.32235757 +0000 UTC m=+1763.250219179" watchObservedRunningTime="2026-07-08 03:09:35.325062339 +0000 UTC m=+1763.252923959"
Jul 08 03:09:41.316133 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:41.316102    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:42.045121 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.045087    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"]
Jul 08 03:09:42.045307 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.045282    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf" podUID="ef4e6ff6-8fdf-4019-bf08-be969178bec9" containerName="maas-api" containerID="cri-o://83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913" gracePeriod=30
Jul 08 03:09:42.282972 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.282952    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:42.340207 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.340135    2558 generic.go:356] "Generic (PLEG): container finished" podID="ef4e6ff6-8fdf-4019-bf08-be969178bec9" containerID="83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913" exitCode=0
Jul 08 03:09:42.340207 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.340177    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"
Jul 08 03:09:42.340207 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.340201    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf" event={"ID":"ef4e6ff6-8fdf-4019-bf08-be969178bec9","Type":"ContainerDied","Data":"83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913"}
Jul 08 03:09:42.340627 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.340246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf" event={"ID":"ef4e6ff6-8fdf-4019-bf08-be969178bec9","Type":"ContainerDied","Data":"40ff63d96678396097ff124a19333ae1788169557555a46b059f9850574e7fc4"}
Jul 08 03:09:42.340627 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.340267    2558 scope.go:117] "RemoveContainer" containerID="83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913"
Jul 08 03:09:42.348334 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.348316    2558 scope.go:117] "RemoveContainer" containerID="83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913"
Jul 08 03:09:42.348572 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:42.348554    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913\": container with ID starting with 83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913 not found: ID does not exist" containerID="83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913"
Jul 08 03:09:42.348616 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.348582    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913"} err="failed to get container status \"83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913\": rpc error: code = NotFound desc = could not find container \"83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913\": container with ID starting with 83ad196f8992521fb85e92178a7e6002b783136e7f3cb8904a12f64423ff0913 not found: ID does not exist"
Jul 08 03:09:42.419468 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.419441    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef4e6ff6-8fdf-4019-bf08-be969178bec9-maas-api-tls\") pod \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") "
Jul 08 03:09:42.419579 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.419533    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6p6d\" (UniqueName: \"kubernetes.io/projected/ef4e6ff6-8fdf-4019-bf08-be969178bec9-kube-api-access-x6p6d\") pod \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") "
Jul 08 03:09:42.419579 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.419561    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef4e6ff6-8fdf-4019-bf08-be969178bec9-openshift-service-ca\") pod \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\" (UID: \"ef4e6ff6-8fdf-4019-bf08-be969178bec9\") "
Jul 08 03:09:42.419953 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.419932    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef4e6ff6-8fdf-4019-bf08-be969178bec9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef4e6ff6-8fdf-4019-bf08-be969178bec9" (UID: "ef4e6ff6-8fdf-4019-bf08-be969178bec9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:09:42.421435 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.421414    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef4e6ff6-8fdf-4019-bf08-be969178bec9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef4e6ff6-8fdf-4019-bf08-be969178bec9" (UID: "ef4e6ff6-8fdf-4019-bf08-be969178bec9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:09:42.421614 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.421600    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef4e6ff6-8fdf-4019-bf08-be969178bec9-kube-api-access-x6p6d" (OuterVolumeSpecName: "kube-api-access-x6p6d") pod "ef4e6ff6-8fdf-4019-bf08-be969178bec9" (UID: "ef4e6ff6-8fdf-4019-bf08-be969178bec9"). InnerVolumeSpecName "kube-api-access-x6p6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:09:42.520255 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.520232    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef4e6ff6-8fdf-4019-bf08-be969178bec9-maas-api-tls\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:42.520255 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.520253    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6p6d\" (UniqueName: \"kubernetes.io/projected/ef4e6ff6-8fdf-4019-bf08-be969178bec9-kube-api-access-x6p6d\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:42.520376 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.520270    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef4e6ff6-8fdf-4019-bf08-be969178bec9-openshift-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:42.656395 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.656360    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"]
Jul 08 03:09:42.658336 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:42.658312    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32f16294-69677f6f-4nmqf"]
Jul 08 03:09:44.619174 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:44.619141    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef4e6ff6-8fdf-4019-bf08-be969178bec9" path="/var/lib/kubelet/pods/ef4e6ff6-8fdf-4019-bf08-be969178bec9/volumes"
Jul 08 03:09:50.415715 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.415683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"]
Jul 08 03:09:50.416225 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.416139    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef4e6ff6-8fdf-4019-bf08-be969178bec9" containerName="maas-api"
Jul 08 03:09:50.416225 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.416157    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef4e6ff6-8fdf-4019-bf08-be969178bec9" containerName="maas-api"
Jul 08 03:09:50.416337 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.416236    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef4e6ff6-8fdf-4019-bf08-be969178bec9" containerName="maas-api"
Jul 08 03:09:50.420495 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.420474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.423541 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.423523    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f1ce6bf4-serving-cert\""
Jul 08 03:09:50.437183 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.437159    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"]
Jul 08 03:09:50.473973 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.473943    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.474100 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.473983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzf75\" (UniqueName: \"kubernetes.io/projected/723ba146-5c31-46d3-806d-128143f87663-kube-api-access-rzf75\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.474100 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.474022    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723ba146-5c31-46d3-806d-128143f87663-openshift-service-ca\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.574652 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.574621    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.574803 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.574664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzf75\" (UniqueName: \"kubernetes.io/projected/723ba146-5c31-46d3-806d-128143f87663-kube-api-access-rzf75\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.574803 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.574703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723ba146-5c31-46d3-806d-128143f87663-openshift-service-ca\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.574803 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:50.574786    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-serving-cert: secret "maas-api-e2e-derive-f1ce6bf4-serving-cert" not found
Jul 08 03:09:50.574942 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:50.574916    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls podName:723ba146-5c31-46d3-806d-128143f87663 nodeName:}" failed. No retries permitted until 2026-07-08 03:09:51.074893937 +0000 UTC m=+1779.002755527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls") pod "maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" (UID: "723ba146-5c31-46d3-806d-128143f87663") : secret "maas-api-e2e-derive-f1ce6bf4-serving-cert" not found
Jul 08 03:09:50.575338 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.575321    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723ba146-5c31-46d3-806d-128143f87663-openshift-service-ca\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:50.589200 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:50.589167    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzf75\" (UniqueName: \"kubernetes.io/projected/723ba146-5c31-46d3-806d-128143f87663-kube-api-access-rzf75\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:51.078192 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:51.078159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:51.080580 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:51.080551    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls\") pod \"maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:51.330577 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:51.330482    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:51.466288 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:51.462256    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"]
Jul 08 03:09:52.391826 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:52.391778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" event={"ID":"723ba146-5c31-46d3-806d-128143f87663","Type":"ContainerStarted","Data":"6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa"}
Jul 08 03:09:52.392031 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:52.391897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" event={"ID":"723ba146-5c31-46d3-806d-128143f87663","Type":"ContainerStarted","Data":"f7fbe8720827dc1ff2b423a161abe91e22decb684760f68d2406be32d8ab184d"}
Jul 08 03:09:52.392031 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:52.391924    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:52.412044 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:52.411998    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" podStartSLOduration=2.026614458 podStartE2EDuration="2.411983175s" totalImagesPullingTime="385.368717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:09:52.410355719 +0000 UTC m=+1780.338217341" watchObservedRunningTime="2026-07-08 03:09:52.411983175 +0000 UTC m=+1780.339844784"
Jul 08 03:09:57.625162 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.625132    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"]
Jul 08 03:09:57.625603 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.625334    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" podUID="723ba146-5c31-46d3-806d-128143f87663" containerName="maas-api" containerID="cri-o://6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa" gracePeriod=30
Jul 08 03:09:57.631033 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.631011    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:57.870780 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.870754    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:57.929533 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.929506    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls\") pod \"723ba146-5c31-46d3-806d-128143f87663\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") "
Jul 08 03:09:57.929695 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.929547    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzf75\" (UniqueName: \"kubernetes.io/projected/723ba146-5c31-46d3-806d-128143f87663-kube-api-access-rzf75\") pod \"723ba146-5c31-46d3-806d-128143f87663\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") "
Jul 08 03:09:57.929695 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.929608    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723ba146-5c31-46d3-806d-128143f87663-openshift-service-ca\") pod \"723ba146-5c31-46d3-806d-128143f87663\" (UID: \"723ba146-5c31-46d3-806d-128143f87663\") "
Jul 08 03:09:57.930012 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.929978    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723ba146-5c31-46d3-806d-128143f87663-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "723ba146-5c31-46d3-806d-128143f87663" (UID: "723ba146-5c31-46d3-806d-128143f87663"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:09:57.931699 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.931675    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "723ba146-5c31-46d3-806d-128143f87663" (UID: "723ba146-5c31-46d3-806d-128143f87663"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:09:57.931699 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:57.931685    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723ba146-5c31-46d3-806d-128143f87663-kube-api-access-rzf75" (OuterVolumeSpecName: "kube-api-access-rzf75") pod "723ba146-5c31-46d3-806d-128143f87663" (UID: "723ba146-5c31-46d3-806d-128143f87663"). InnerVolumeSpecName "kube-api-access-rzf75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:09:58.030528 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.030499    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723ba146-5c31-46d3-806d-128143f87663-openshift-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:58.030528 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.030522    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723ba146-5c31-46d3-806d-128143f87663-maas-api-tls\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:58.030528 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.030532    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzf75\" (UniqueName: \"kubernetes.io/projected/723ba146-5c31-46d3-806d-128143f87663-kube-api-access-rzf75\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:09:58.420140 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.420112    2558 generic.go:356] "Generic (PLEG): container finished" podID="723ba146-5c31-46d3-806d-128143f87663" containerID="6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa" exitCode=0
Jul 08 03:09:58.420302 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.420155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" event={"ID":"723ba146-5c31-46d3-806d-128143f87663","Type":"ContainerDied","Data":"6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa"}
Jul 08 03:09:58.420302 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.420162    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"
Jul 08 03:09:58.420302 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.420191    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d" event={"ID":"723ba146-5c31-46d3-806d-128143f87663","Type":"ContainerDied","Data":"f7fbe8720827dc1ff2b423a161abe91e22decb684760f68d2406be32d8ab184d"}
Jul 08 03:09:58.420302 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.420211    2558 scope.go:117] "RemoveContainer" containerID="6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa"
Jul 08 03:09:58.429118 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.429100    2558 scope.go:117] "RemoveContainer" containerID="6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa"
Jul 08 03:09:58.429358 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:09:58.429340    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa\": container with ID starting with 6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa not found: ID does not exist" containerID="6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa"
Jul 08 03:09:58.429403 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.429366    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa"} err="failed to get container status \"6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa\": rpc error: code = NotFound desc = could not find container \"6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa\": container with ID starting with 6300fbf4fb03233c6260496740df5059c8d9db408d42a04c2b0a124063c005aa not found: ID does not exist"
Jul 08 03:09:58.441026 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.440998    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"]
Jul 08 03:09:58.447139 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.447119    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1ce6bf4-75b4ff9d76-bkn7d"]
Jul 08 03:09:58.619033 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:09:58.618995    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723ba146-5c31-46d3-806d-128143f87663" path="/var/lib/kubelet/pods/723ba146-5c31-46d3-806d-128143f87663/volumes"
Jul 08 03:10:12.609376 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:10:12.609346    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:10:12.618386 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:10:12.618356    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:14:38.250787 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.250714    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"]
Jul 08 03:14:38.253081 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.251040    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723ba146-5c31-46d3-806d-128143f87663" containerName="maas-api"
Jul 08 03:14:38.253081 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.251053    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="723ba146-5c31-46d3-806d-128143f87663" containerName="maas-api"
Jul 08 03:14:38.253081 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.251114    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="723ba146-5c31-46d3-806d-128143f87663" containerName="maas-api"
Jul 08 03:14:38.253951 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.253936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.256605 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.256583    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c99a305e\"/\"default-dockercfg-qzkkc\""
Jul 08 03:14:38.256724 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.256654    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c99a305e\"/\"openshift-service-ca.crt\""
Jul 08 03:14:38.257697 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.257684    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c99a305e\"/\"kube-root-ca.crt\""
Jul 08 03:14:38.257765 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.257749    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c99a305e\"/\"e2e-lifecycle-model-c99a305e-kserve-self-signed-certs\""
Jul 08 03:14:38.263193 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.263171    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"]
Jul 08 03:14:38.391688 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.391655    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-dshm\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.391876 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.391702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060d480d-c5d4-4b11-a89a-8ca004396e6a-tls-certs\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.391876 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.391732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-tmp-dir\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.391876 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.391794    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-model-cache\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.391876 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.391834    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-home\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.392048 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.391916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2kx\" (UniqueName: \"kubernetes.io/projected/060d480d-c5d4-4b11-a89a-8ca004396e6a-kube-api-access-cs2kx\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492563 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060d480d-c5d4-4b11-a89a-8ca004396e6a-tls-certs\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492563 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-tmp-dir\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492718 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492586    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-model-cache\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492756 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492709    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-home\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492790 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2kx\" (UniqueName: \"kubernetes.io/projected/060d480d-c5d4-4b11-a89a-8ca004396e6a-kube-api-access-cs2kx\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492822 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-dshm\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.492974 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492955    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-model-cache\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.493016 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.492997    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-tmp-dir\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.493076 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.493056    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-home\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.495033 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.495009    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-dshm\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.495228 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.495212    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060d480d-c5d4-4b11-a89a-8ca004396e6a-tls-certs\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.500537 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.500518    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2kx\" (UniqueName: \"kubernetes.io/projected/060d480d-c5d4-4b11-a89a-8ca004396e6a-kube-api-access-cs2kx\") pod \"e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") " pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.563604 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.563549    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:38.693833 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.693809    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"]
Jul 08 03:14:38.695665 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:14:38.695640    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod060d480d_c5d4_4b11_a89a_8ca004396e6a.slice/crio-140f90977b19e8fd8a3898dfecc431451cea4814918b4a2509b0144036d71de4 WatchSource:0}: Error finding container 140f90977b19e8fd8a3898dfecc431451cea4814918b4a2509b0144036d71de4: Status 404 returned error can't find the container with id 140f90977b19e8fd8a3898dfecc431451cea4814918b4a2509b0144036d71de4
Jul 08 03:14:38.714105 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:38.714081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2" event={"ID":"060d480d-c5d4-4b11-a89a-8ca004396e6a","Type":"ContainerStarted","Data":"140f90977b19e8fd8a3898dfecc431451cea4814918b4a2509b0144036d71de4"}
Jul 08 03:14:39.720946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:39.720912    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2" event={"ID":"060d480d-c5d4-4b11-a89a-8ca004396e6a","Type":"ContainerStarted","Data":"868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892"}
Jul 08 03:14:39.738406 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:39.738359    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2" podStartSLOduration=1.7383448769999998 podStartE2EDuration="1.738344877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:14:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:14:39.73764598 +0000 UTC m=+2067.665507591" watchObservedRunningTime="2026-07-08 03:14:39.738344877 +0000 UTC m=+2067.666206485"
Jul 08 03:14:48.564476 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:48.564442    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:48.564476 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:48.564486    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:48.576762 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:48.576739    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:14:48.772998 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:14:48.772965    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:15:12.638807 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:12.638779    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:15:12.647525 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:12.647502    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:15:22.208808 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:22.208773    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"]
Jul 08 03:15:37.231268 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.231205    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2" podUID="060d480d-c5d4-4b11-a89a-8ca004396e6a" containerName="main" containerID="cri-o://868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892" gracePeriod=45
Jul 08 03:15:37.467808 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.467781    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:15:37.576502 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576426    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-dshm\") pod \"060d480d-c5d4-4b11-a89a-8ca004396e6a\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") "
Jul 08 03:15:37.576502 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576459    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060d480d-c5d4-4b11-a89a-8ca004396e6a-tls-certs\") pod \"060d480d-c5d4-4b11-a89a-8ca004396e6a\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") "
Jul 08 03:15:37.576708 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576535    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-model-cache\") pod \"060d480d-c5d4-4b11-a89a-8ca004396e6a\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") "
Jul 08 03:15:37.576708 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576560    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cs2kx\" (UniqueName: \"kubernetes.io/projected/060d480d-c5d4-4b11-a89a-8ca004396e6a-kube-api-access-cs2kx\") pod \"060d480d-c5d4-4b11-a89a-8ca004396e6a\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") "
Jul 08 03:15:37.576708 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576621    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-tmp-dir\") pod \"060d480d-c5d4-4b11-a89a-8ca004396e6a\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") "
Jul 08 03:15:37.576708 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576641    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-home\") pod \"060d480d-c5d4-4b11-a89a-8ca004396e6a\" (UID: \"060d480d-c5d4-4b11-a89a-8ca004396e6a\") "
Jul 08 03:15:37.576947 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576879    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-model-cache" (OuterVolumeSpecName: "model-cache") pod "060d480d-c5d4-4b11-a89a-8ca004396e6a" (UID: "060d480d-c5d4-4b11-a89a-8ca004396e6a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:15:37.577009 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576936    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "060d480d-c5d4-4b11-a89a-8ca004396e6a" (UID: "060d480d-c5d4-4b11-a89a-8ca004396e6a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:15:37.577009 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.576981    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-home" (OuterVolumeSpecName: "home") pod "060d480d-c5d4-4b11-a89a-8ca004396e6a" (UID: "060d480d-c5d4-4b11-a89a-8ca004396e6a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:15:37.578749 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.578723    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/060d480d-c5d4-4b11-a89a-8ca004396e6a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "060d480d-c5d4-4b11-a89a-8ca004396e6a" (UID: "060d480d-c5d4-4b11-a89a-8ca004396e6a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:15:37.578823 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.578766    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/060d480d-c5d4-4b11-a89a-8ca004396e6a-kube-api-access-cs2kx" (OuterVolumeSpecName: "kube-api-access-cs2kx") pod "060d480d-c5d4-4b11-a89a-8ca004396e6a" (UID: "060d480d-c5d4-4b11-a89a-8ca004396e6a"). InnerVolumeSpecName "kube-api-access-cs2kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:15:37.578823 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.578799    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-dshm" (OuterVolumeSpecName: "dshm") pod "060d480d-c5d4-4b11-a89a-8ca004396e6a" (UID: "060d480d-c5d4-4b11-a89a-8ca004396e6a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:15:37.677481 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.677449    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-model-cache\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:15:37.677481 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.677478    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs2kx\" (UniqueName: \"kubernetes.io/projected/060d480d-c5d4-4b11-a89a-8ca004396e6a-kube-api-access-cs2kx\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:15:37.677644 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.677493    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-tmp-dir\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:15:37.677644 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.677505    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-home\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:15:37.677644 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.677515    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060d480d-c5d4-4b11-a89a-8ca004396e6a-dshm\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:15:37.677644 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.677524    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060d480d-c5d4-4b11-a89a-8ca004396e6a-tls-certs\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:15:37.990501 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.990470    2558 generic.go:356] "Generic (PLEG): container finished" podID="060d480d-c5d4-4b11-a89a-8ca004396e6a" containerID="868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892" exitCode=0
Jul 08 03:15:37.990690 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.990512    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"
Jul 08 03:15:37.990690 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.990558    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2" event={"ID":"060d480d-c5d4-4b11-a89a-8ca004396e6a","Type":"ContainerDied","Data":"868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892"}
Jul 08 03:15:37.990690 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.990603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2" event={"ID":"060d480d-c5d4-4b11-a89a-8ca004396e6a","Type":"ContainerDied","Data":"140f90977b19e8fd8a3898dfecc431451cea4814918b4a2509b0144036d71de4"}
Jul 08 03:15:37.990690 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.990624    2558 scope.go:117] "RemoveContainer" containerID="868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892"
Jul 08 03:15:37.999489 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.999475    2558 scope.go:117] "RemoveContainer" containerID="868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892"
Jul 08 03:15:37.999743 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:15:37.999726    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892\": container with ID starting with 868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892 not found: ID does not exist" containerID="868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892"
Jul 08 03:15:37.999794 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:37.999750    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892"} err="failed to get container status \"868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892\": rpc error: code = NotFound desc = could not find container \"868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892\": container with ID starting with 868b882f394e2c1a72ff7add998e88a8bfd5fe6bc76275dbf03f1a3667a11892 not found: ID does not exist"
Jul 08 03:15:38.010747 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:38.010729    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"]
Jul 08 03:15:38.013927 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:38.013909    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c99a305e/e2e-lifecycle-model-c99a305e-kserve-cc4776fcf-rwrd2"]
Jul 08 03:15:38.619534 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:15:38.619493    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="060d480d-c5d4-4b11-a89a-8ca004396e6a" path="/var/lib/kubelet/pods/060d480d-c5d4-4b11-a89a-8ca004396e6a/volumes"
Jul 08 03:18:19.079355 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.079320    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"]
Jul 08 03:18:19.079822 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.079642    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="060d480d-c5d4-4b11-a89a-8ca004396e6a" containerName="main"
Jul 08 03:18:19.079822 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.079652    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="060d480d-c5d4-4b11-a89a-8ca004396e6a" containerName="main"
Jul 08 03:18:19.079822 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.079702    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="060d480d-c5d4-4b11-a89a-8ca004396e6a" containerName="main"
Jul 08 03:18:19.082565 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.082549    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.085411 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.085385    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-91d71d-serving-cert\""
Jul 08 03:18:19.095039 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.095015    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"]
Jul 08 03:18:19.182339 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.182311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.182339 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.182341    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xvtz\" (UniqueName: \"kubernetes.io/projected/81c69a6d-b378-4470-89c7-78ddaf83a847-kube-api-access-8xvtz\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.182554 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.182369    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c69a6d-b378-4470-89c7-78ddaf83a847-openshift-service-ca\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.283686 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.283654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.283686 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.283685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xvtz\" (UniqueName: \"kubernetes.io/projected/81c69a6d-b378-4470-89c7-78ddaf83a847-kube-api-access-8xvtz\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.283925 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.283715    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c69a6d-b378-4470-89c7-78ddaf83a847-openshift-service-ca\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.283925 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:18:19.283864    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-serving-cert: secret "maas-api-e2e-inf-a-91d71d-serving-cert" not found
Jul 08 03:18:19.283997 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:18:19.283936    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls podName:81c69a6d-b378-4470-89c7-78ddaf83a847 nodeName:}" failed. No retries permitted until 2026-07-08 03:18:19.783916614 +0000 UTC m=+2287.711778201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls") pod "maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" (UID: "81c69a6d-b378-4470-89c7-78ddaf83a847") : secret "maas-api-e2e-inf-a-91d71d-serving-cert" not found
Jul 08 03:18:19.284361 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.284343    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c69a6d-b378-4470-89c7-78ddaf83a847-openshift-service-ca\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.301863 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.301814    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xvtz\" (UniqueName: \"kubernetes.io/projected/81c69a6d-b378-4470-89c7-78ddaf83a847-kube-api-access-8xvtz\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.787260 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.787226    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.789637 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.789612    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls\") pod \"maas-api-e2e-inf-a-91d71d-ddc758895-64pcn\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:19.992762 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:19.992733    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:20.129976 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:20.129949    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"]
Jul 08 03:18:20.132686 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:18:20.132655    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c69a6d_b378_4470_89c7_78ddaf83a847.slice/crio-795d594895ef9abaaa75ee6140a8c3018cea223945999acb1e52619447323f3c WatchSource:0}: Error finding container 795d594895ef9abaaa75ee6140a8c3018cea223945999acb1e52619447323f3c: Status 404 returned error can't find the container with id 795d594895ef9abaaa75ee6140a8c3018cea223945999acb1e52619447323f3c
Jul 08 03:18:20.134233 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:20.134213    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:18:20.723798 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:20.723763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" event={"ID":"81c69a6d-b378-4470-89c7-78ddaf83a847","Type":"ContainerStarted","Data":"795d594895ef9abaaa75ee6140a8c3018cea223945999acb1e52619447323f3c"}
Jul 08 03:18:21.730359 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:21.730262    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" event={"ID":"81c69a6d-b378-4470-89c7-78ddaf83a847","Type":"ContainerStarted","Data":"5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093"}
Jul 08 03:18:21.730359 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:21.730345    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:18:21.747987 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:21.747934    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" podStartSLOduration=1.410345378 podStartE2EDuration="2.747920717s" totalImagesPullingTime="1.337575339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:18:21.745993593 +0000 UTC m=+2289.673855195" watchObservedRunningTime="2026-07-08 03:18:21.747920717 +0000 UTC m=+2289.675782317"
Jul 08 03:18:27.741140 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:18:27.741101    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:19:14.656677 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.656644    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"]
Jul 08 03:19:14.660011 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.659996    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.662442 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.662419    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a6e00a\"/\"kube-root-ca.crt\""
Jul 08 03:19:14.662570 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.662467    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a6e00a\"/\"openshift-service-ca.crt\""
Jul 08 03:19:14.663543 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.663526    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a6e00a\"/\"default-dockercfg-f4w9n\""
Jul 08 03:19:14.663653 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.663577    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a6e00a\"/\"test-model-a6e00a-kserve-self-signed-certs\""
Jul 08 03:19:14.668535 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.668506    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"]
Jul 08 03:19:14.761576 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.761540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-tmp-dir\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.761576 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.761578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-model-cache\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.761762 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.761597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2dc888d-6955-47a1-9103-53404e921448-tls-certs\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.761762 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.761614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-dshm\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.761762 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.761719    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfsfn\" (UniqueName: \"kubernetes.io/projected/b2dc888d-6955-47a1-9103-53404e921448-kube-api-access-wfsfn\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.761876 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.761774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-home\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.862599 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.862566    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2dc888d-6955-47a1-9103-53404e921448-tls-certs\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.862737 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.862606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-dshm\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.862798 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.862783    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfsfn\" (UniqueName: \"kubernetes.io/projected/b2dc888d-6955-47a1-9103-53404e921448-kube-api-access-wfsfn\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.862890 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.862881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-home\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.862965 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.862951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-tmp-dir\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.863017 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.862984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-model-cache\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.863244 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.863221    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-home\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.863479 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.863273    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-tmp-dir\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.863536 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.863489    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-model-cache\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.864952 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.864933    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-dshm\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.865193 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.865178    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2dc888d-6955-47a1-9103-53404e921448-tls-certs\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.870645 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.870624    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfsfn\" (UniqueName: \"kubernetes.io/projected/b2dc888d-6955-47a1-9103-53404e921448-kube-api-access-wfsfn\") pod \"test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") " pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:14.969669 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:14.969646    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:15.098202 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:15.098170    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"]
Jul 08 03:19:15.099852 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:19:15.099807    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2dc888d_6955_47a1_9103_53404e921448.slice/crio-84777a5c423d400b669c8c1c58fa3ea07afb088e43ff55b2bafa633a79ef15ae WatchSource:0}: Error finding container 84777a5c423d400b669c8c1c58fa3ea07afb088e43ff55b2bafa633a79ef15ae: Status 404 returned error can't find the container with id 84777a5c423d400b669c8c1c58fa3ea07afb088e43ff55b2bafa633a79ef15ae
Jul 08 03:19:15.976657 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:15.976615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl" event={"ID":"b2dc888d-6955-47a1-9103-53404e921448","Type":"ContainerStarted","Data":"a89f2cb46a3675ccf60a6946848ca6b73c2ee1dd4694c52fb6c1da80a54edf18"}
Jul 08 03:19:15.977044 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:15.976667    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl" event={"ID":"b2dc888d-6955-47a1-9103-53404e921448","Type":"ContainerStarted","Data":"84777a5c423d400b669c8c1c58fa3ea07afb088e43ff55b2bafa633a79ef15ae"}
Jul 08 03:19:15.994258 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:15.994212    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl" podStartSLOduration=1.9941972030000001 podStartE2EDuration="1.994197203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:19:15.993291306 +0000 UTC m=+2343.921152917" watchObservedRunningTime="2026-07-08 03:19:15.994197203 +0000 UTC m=+2343.922058810"
Jul 08 03:19:24.970706 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:24.970652    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:24.970706 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:24.970722    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:24.982899 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:24.982878    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:19:25.028648 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:19:25.028620    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:20:12.675029 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:20:12.674997    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:20:12.685868 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:20:12.685831    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:21:10.443312 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:10.443272    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"]
Jul 08 03:21:11.496603 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.496567    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"]
Jul 08 03:21:11.497126 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.496802    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" podUID="81c69a6d-b378-4470-89c7-78ddaf83a847" containerName="maas-api" containerID="cri-o://5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093" gracePeriod=30
Jul 08 03:21:11.735950 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.735928    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:21:11.851484 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.851407    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls\") pod \"81c69a6d-b378-4470-89c7-78ddaf83a847\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") "
Jul 08 03:21:11.851484 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.851449    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c69a6d-b378-4470-89c7-78ddaf83a847-openshift-service-ca\") pod \"81c69a6d-b378-4470-89c7-78ddaf83a847\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") "
Jul 08 03:21:11.851663 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.851582    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xvtz\" (UniqueName: \"kubernetes.io/projected/81c69a6d-b378-4470-89c7-78ddaf83a847-kube-api-access-8xvtz\") pod \"81c69a6d-b378-4470-89c7-78ddaf83a847\" (UID: \"81c69a6d-b378-4470-89c7-78ddaf83a847\") "
Jul 08 03:21:11.851820 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.851798    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c69a6d-b378-4470-89c7-78ddaf83a847-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81c69a6d-b378-4470-89c7-78ddaf83a847" (UID: "81c69a6d-b378-4470-89c7-78ddaf83a847"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:21:11.853522 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.853497    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81c69a6d-b378-4470-89c7-78ddaf83a847" (UID: "81c69a6d-b378-4470-89c7-78ddaf83a847"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:21:11.853611 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.853591    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c69a6d-b378-4470-89c7-78ddaf83a847-kube-api-access-8xvtz" (OuterVolumeSpecName: "kube-api-access-8xvtz") pod "81c69a6d-b378-4470-89c7-78ddaf83a847" (UID: "81c69a6d-b378-4470-89c7-78ddaf83a847"). InnerVolumeSpecName "kube-api-access-8xvtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:21:11.953177 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.953151    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c69a6d-b378-4470-89c7-78ddaf83a847-maas-api-tls\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:11.953177 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.953175    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c69a6d-b378-4470-89c7-78ddaf83a847-openshift-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:11.953324 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:11.953184    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xvtz\" (UniqueName: \"kubernetes.io/projected/81c69a6d-b378-4470-89c7-78ddaf83a847-kube-api-access-8xvtz\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:12.532248 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.532213    2558 generic.go:356] "Generic (PLEG): container finished" podID="81c69a6d-b378-4470-89c7-78ddaf83a847" containerID="5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093" exitCode=0
Jul 08 03:21:12.532248 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.532248    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"
Jul 08 03:21:12.532722 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.532262    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" event={"ID":"81c69a6d-b378-4470-89c7-78ddaf83a847","Type":"ContainerDied","Data":"5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093"}
Jul 08 03:21:12.532722 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.532304    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn" event={"ID":"81c69a6d-b378-4470-89c7-78ddaf83a847","Type":"ContainerDied","Data":"795d594895ef9abaaa75ee6140a8c3018cea223945999acb1e52619447323f3c"}
Jul 08 03:21:12.532722 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.532324    2558 scope.go:117] "RemoveContainer" containerID="5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093"
Jul 08 03:21:12.541600 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.541583    2558 scope.go:117] "RemoveContainer" containerID="5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093"
Jul 08 03:21:12.541917 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:21:12.541897    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093\": container with ID starting with 5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093 not found: ID does not exist" containerID="5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093"
Jul 08 03:21:12.541969 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.541926    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093"} err="failed to get container status \"5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093\": rpc error: code = NotFound desc = could not find container \"5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093\": container with ID starting with 5f8db07afdced08cef7a54a9fab23142a92ba541462d9aabc0558e6d94ac7093 not found: ID does not exist"
Jul 08 03:21:12.553987 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.553964    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"]
Jul 08 03:21:12.557711 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.557692    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-91d71d-ddc758895-64pcn"]
Jul 08 03:21:12.619524 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:12.619498    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81c69a6d-b378-4470-89c7-78ddaf83a847" path="/var/lib/kubelet/pods/81c69a6d-b378-4470-89c7-78ddaf83a847/volumes"
Jul 08 03:21:25.463392 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.463353    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl" podUID="b2dc888d-6955-47a1-9103-53404e921448" containerName="main" containerID="cri-o://a89f2cb46a3675ccf60a6946848ca6b73c2ee1dd4694c52fb6c1da80a54edf18" gracePeriod=45
Jul 08 03:21:25.594323 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.594279    2558 generic.go:356] "Generic (PLEG): container finished" podID="b2dc888d-6955-47a1-9103-53404e921448" containerID="a89f2cb46a3675ccf60a6946848ca6b73c2ee1dd4694c52fb6c1da80a54edf18" exitCode=0
Jul 08 03:21:25.594456 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.594343    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl" event={"ID":"b2dc888d-6955-47a1-9103-53404e921448","Type":"ContainerDied","Data":"a89f2cb46a3675ccf60a6946848ca6b73c2ee1dd4694c52fb6c1da80a54edf18"}
Jul 08 03:21:25.701681 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.701658    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:21:25.763998 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.763930    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-tmp-dir\") pod \"b2dc888d-6955-47a1-9103-53404e921448\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") "
Jul 08 03:21:25.764125 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.763999    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-model-cache\") pod \"b2dc888d-6955-47a1-9103-53404e921448\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") "
Jul 08 03:21:25.764125 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-dshm\") pod \"b2dc888d-6955-47a1-9103-53404e921448\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") "
Jul 08 03:21:25.764125 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764055    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2dc888d-6955-47a1-9103-53404e921448-tls-certs\") pod \"b2dc888d-6955-47a1-9103-53404e921448\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") "
Jul 08 03:21:25.764125 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764116    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2dc888d-6955-47a1-9103-53404e921448" (UID: "b2dc888d-6955-47a1-9103-53404e921448"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:21:25.764328 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764135    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-home\") pod \"b2dc888d-6955-47a1-9103-53404e921448\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") "
Jul 08 03:21:25.764328 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764165    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfsfn\" (UniqueName: \"kubernetes.io/projected/b2dc888d-6955-47a1-9103-53404e921448-kube-api-access-wfsfn\") pod \"b2dc888d-6955-47a1-9103-53404e921448\" (UID: \"b2dc888d-6955-47a1-9103-53404e921448\") "
Jul 08 03:21:25.764429 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764306    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2dc888d-6955-47a1-9103-53404e921448" (UID: "b2dc888d-6955-47a1-9103-53404e921448"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:21:25.764429 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764366    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-home" (OuterVolumeSpecName: "home") pod "b2dc888d-6955-47a1-9103-53404e921448" (UID: "b2dc888d-6955-47a1-9103-53404e921448"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:21:25.764429 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764380    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-tmp-dir\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:25.764429 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.764391    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-model-cache\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:25.766304 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.766284    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-dshm" (OuterVolumeSpecName: "dshm") pod "b2dc888d-6955-47a1-9103-53404e921448" (UID: "b2dc888d-6955-47a1-9103-53404e921448"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:21:25.766374 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.766299    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2dc888d-6955-47a1-9103-53404e921448-kube-api-access-wfsfn" (OuterVolumeSpecName: "kube-api-access-wfsfn") pod "b2dc888d-6955-47a1-9103-53404e921448" (UID: "b2dc888d-6955-47a1-9103-53404e921448"). InnerVolumeSpecName "kube-api-access-wfsfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:21:25.766374 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.766354    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2dc888d-6955-47a1-9103-53404e921448-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2dc888d-6955-47a1-9103-53404e921448" (UID: "b2dc888d-6955-47a1-9103-53404e921448"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:21:25.865420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.865392    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-home\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:25.865420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.865414    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfsfn\" (UniqueName: \"kubernetes.io/projected/b2dc888d-6955-47a1-9103-53404e921448-kube-api-access-wfsfn\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:25.865420 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.865424    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2dc888d-6955-47a1-9103-53404e921448-dshm\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:25.865603 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:25.865433    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2dc888d-6955-47a1-9103-53404e921448-tls-certs\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:21:26.600539 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:26.600463    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"
Jul 08 03:21:26.600539 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:26.600475    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl" event={"ID":"b2dc888d-6955-47a1-9103-53404e921448","Type":"ContainerDied","Data":"84777a5c423d400b669c8c1c58fa3ea07afb088e43ff55b2bafa633a79ef15ae"}
Jul 08 03:21:26.600539 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:26.600517    2558 scope.go:117] "RemoveContainer" containerID="a89f2cb46a3675ccf60a6946848ca6b73c2ee1dd4694c52fb6c1da80a54edf18"
Jul 08 03:21:26.621198 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:26.621171    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"]
Jul 08 03:21:26.625214 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:26.625194    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a6e00a/test-model-a6e00a-kserve-7d6d9cf5d7-lxpjl"]
Jul 08 03:21:28.618593 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:21:28.618558    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2dc888d-6955-47a1-9103-53404e921448" path="/var/lib/kubelet/pods/b2dc888d-6955-47a1-9103-53404e921448/volumes"
Jul 08 03:22:04.150503 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150427    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"]
Jul 08 03:22:04.150946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150704    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2dc888d-6955-47a1-9103-53404e921448" containerName="main"
Jul 08 03:22:04.150946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150714    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2dc888d-6955-47a1-9103-53404e921448" containerName="main"
Jul 08 03:22:04.150946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150726    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c69a6d-b378-4470-89c7-78ddaf83a847" containerName="maas-api"
Jul 08 03:22:04.150946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150732    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c69a6d-b378-4470-89c7-78ddaf83a847" containerName="maas-api"
Jul 08 03:22:04.150946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150784    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="81c69a6d-b378-4470-89c7-78ddaf83a847" containerName="maas-api"
Jul 08 03:22:04.150946 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.150793    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2dc888d-6955-47a1-9103-53404e921448" containerName="main"
Jul 08 03:22:04.153569 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.153554    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.155795 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.155776    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ed1785-serving-cert\""
Jul 08 03:22:04.162467 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.162438    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"]
Jul 08 03:22:04.263315 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.263279    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pph5\" (UniqueName: \"kubernetes.io/projected/50b5f13f-555f-439c-b119-50e114441fe5-kube-api-access-8pph5\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.263492 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.263401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50b5f13f-555f-439c-b119-50e114441fe5-maas-api-tls\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.263492 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.263473    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b5f13f-555f-439c-b119-50e114441fe5-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.364641 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.364611    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50b5f13f-555f-439c-b119-50e114441fe5-maas-api-tls\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.364790 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.364657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b5f13f-555f-439c-b119-50e114441fe5-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.364790 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.364679    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pph5\" (UniqueName: \"kubernetes.io/projected/50b5f13f-555f-439c-b119-50e114441fe5-kube-api-access-8pph5\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.365354 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.365332    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b5f13f-555f-439c-b119-50e114441fe5-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.367273 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.367247    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50b5f13f-555f-439c-b119-50e114441fe5-maas-api-tls\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.372554 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.372532    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pph5\" (UniqueName: \"kubernetes.io/projected/50b5f13f-555f-439c-b119-50e114441fe5-kube-api-access-8pph5\") pod \"maas-api-e2e-api-a-ed1785-c7574d748-p5v9b\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.463493 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.463470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:04.597875 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.592465    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"]
Jul 08 03:22:04.772474 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:04.772396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b" event={"ID":"50b5f13f-555f-439c-b119-50e114441fe5","Type":"ContainerStarted","Data":"91d6319e179a95b163bd94a153cba39c95fcd113539568ba700354c7b2c4f77c"}
Jul 08 03:22:05.778062 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:05.778025    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b" event={"ID":"50b5f13f-555f-439c-b119-50e114441fe5","Type":"ContainerStarted","Data":"d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd"}
Jul 08 03:22:05.778454 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:05.778156    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:05.796221 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:05.796174    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b" podStartSLOduration=1.309341824 podStartE2EDuration="1.79615927s" totalImagesPullingTime="486.817446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:22:05.79390659 +0000 UTC m=+2513.721768199" watchObservedRunningTime="2026-07-08 03:22:05.79615927 +0000 UTC m=+2513.724020879"
Jul 08 03:22:11.788912 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:11.788885    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:25.519976 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.519940    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"]
Jul 08 03:22:25.520483 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.520187    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b" podUID="50b5f13f-555f-439c-b119-50e114441fe5" containerName="maas-api" containerID="cri-o://d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd" gracePeriod=30
Jul 08 03:22:25.766695 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.766669    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:25.871657 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.871576    2558 generic.go:356] "Generic (PLEG): container finished" podID="50b5f13f-555f-439c-b119-50e114441fe5" containerID="d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd" exitCode=0
Jul 08 03:22:25.871657 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.871621    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"
Jul 08 03:22:25.871821 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.871625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b" event={"ID":"50b5f13f-555f-439c-b119-50e114441fe5","Type":"ContainerDied","Data":"d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd"}
Jul 08 03:22:25.871821 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.871777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b" event={"ID":"50b5f13f-555f-439c-b119-50e114441fe5","Type":"ContainerDied","Data":"91d6319e179a95b163bd94a153cba39c95fcd113539568ba700354c7b2c4f77c"}
Jul 08 03:22:25.871821 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.871804    2558 scope.go:117] "RemoveContainer" containerID="d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd"
Jul 08 03:22:25.880005 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.879986    2558 scope.go:117] "RemoveContainer" containerID="d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd"
Jul 08 03:22:25.880259 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:22:25.880242    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd\": container with ID starting with d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd not found: ID does not exist" containerID="d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd"
Jul 08 03:22:25.880304 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.880267    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd"} err="failed to get container status \"d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd\": rpc error: code = NotFound desc = could not find container \"d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd\": container with ID starting with d9fe5488804f00db29879827c8535fcb2093e8092511d9b3c61bb1a1590490bd not found: ID does not exist"
Jul 08 03:22:25.942151 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.942127    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b5f13f-555f-439c-b119-50e114441fe5-openshift-service-ca\") pod \"50b5f13f-555f-439c-b119-50e114441fe5\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") "
Jul 08 03:22:25.942245 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.942167    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pph5\" (UniqueName: \"kubernetes.io/projected/50b5f13f-555f-439c-b119-50e114441fe5-kube-api-access-8pph5\") pod \"50b5f13f-555f-439c-b119-50e114441fe5\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") "
Jul 08 03:22:25.942245 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.942219    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50b5f13f-555f-439c-b119-50e114441fe5-maas-api-tls\") pod \"50b5f13f-555f-439c-b119-50e114441fe5\" (UID: \"50b5f13f-555f-439c-b119-50e114441fe5\") "
Jul 08 03:22:25.942526 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.942499    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50b5f13f-555f-439c-b119-50e114441fe5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50b5f13f-555f-439c-b119-50e114441fe5" (UID: "50b5f13f-555f-439c-b119-50e114441fe5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:22:25.944375 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.944352    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50b5f13f-555f-439c-b119-50e114441fe5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50b5f13f-555f-439c-b119-50e114441fe5" (UID: "50b5f13f-555f-439c-b119-50e114441fe5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:22:25.944472 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:25.944375    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50b5f13f-555f-439c-b119-50e114441fe5-kube-api-access-8pph5" (OuterVolumeSpecName: "kube-api-access-8pph5") pod "50b5f13f-555f-439c-b119-50e114441fe5" (UID: "50b5f13f-555f-439c-b119-50e114441fe5"). InnerVolumeSpecName "kube-api-access-8pph5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:22:26.043649 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:26.043623    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pph5\" (UniqueName: \"kubernetes.io/projected/50b5f13f-555f-439c-b119-50e114441fe5-kube-api-access-8pph5\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:22:26.043649 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:26.043648    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50b5f13f-555f-439c-b119-50e114441fe5-maas-api-tls\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:22:26.043819 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:26.043657    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b5f13f-555f-439c-b119-50e114441fe5-openshift-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:22:26.194690 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:26.194661    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"]
Jul 08 03:22:26.197732 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:26.197710    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed1785-c7574d748-p5v9b"]
Jul 08 03:22:26.619832 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:26.619751    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50b5f13f-555f-439c-b119-50e114441fe5" path="/var/lib/kubelet/pods/50b5f13f-555f-439c-b119-50e114441fe5/volumes"
Jul 08 03:22:49.367791 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.367757    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"]
Jul 08 03:22:49.368510 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.368070    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50b5f13f-555f-439c-b119-50e114441fe5" containerName="maas-api"
Jul 08 03:22:49.368510 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.368081    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="50b5f13f-555f-439c-b119-50e114441fe5" containerName="maas-api"
Jul 08 03:22:49.368510 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.368160    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="50b5f13f-555f-439c-b119-50e114441fe5" containerName="maas-api"
Jul 08 03:22:49.371118 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.371100    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.373555 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.373534    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-6493fa-serving-cert\""
Jul 08 03:22:49.382543 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.382518    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"]
Jul 08 03:22:49.436673 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.436644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.436791 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.436688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmj57\" (UniqueName: \"kubernetes.io/projected/04fadd0a-3ad7-4392-b989-6ad11dad9432-kube-api-access-rmj57\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.436791 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.436725    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04fadd0a-3ad7-4392-b989-6ad11dad9432-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.538135 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.538109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.538268 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.538139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmj57\" (UniqueName: \"kubernetes.io/projected/04fadd0a-3ad7-4392-b989-6ad11dad9432-kube-api-access-rmj57\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.538268 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.538169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04fadd0a-3ad7-4392-b989-6ad11dad9432-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.538268 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:22:49.538249    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-serving-cert: secret "maas-api-e2e-shared-b-6493fa-serving-cert" not found
Jul 08 03:22:49.538366 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:22:49.538320    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls podName:04fadd0a-3ad7-4392-b989-6ad11dad9432 nodeName:}" failed. No retries permitted until 2026-07-08 03:22:50.03830511 +0000 UTC m=+2557.966166697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls") pod "maas-api-e2e-shared-b-6493fa-9b956449-p72g8" (UID: "04fadd0a-3ad7-4392-b989-6ad11dad9432") : secret "maas-api-e2e-shared-b-6493fa-serving-cert" not found
Jul 08 03:22:49.538786 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.538769    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04fadd0a-3ad7-4392-b989-6ad11dad9432-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:49.546862 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:49.546818    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmj57\" (UniqueName: \"kubernetes.io/projected/04fadd0a-3ad7-4392-b989-6ad11dad9432-kube-api-access-rmj57\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:50.042076 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.042042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:50.044636 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.044614    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls\") pod \"maas-api-e2e-shared-b-6493fa-9b956449-p72g8\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:50.282008 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.281945    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:50.421909 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.421878    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"]
Jul 08 03:22:50.425144 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:22:50.425118    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fadd0a_3ad7_4392_b989_6ad11dad9432.slice/crio-1fb373a1ebd734f78111d79b30a74d53c7b083ba6d3c81dac232372ad624be2c WatchSource:0}: Error finding container 1fb373a1ebd734f78111d79b30a74d53c7b083ba6d3c81dac232372ad624be2c: Status 404 returned error can't find the container with id 1fb373a1ebd734f78111d79b30a74d53c7b083ba6d3c81dac232372ad624be2c
Jul 08 03:22:50.990592 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.990554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8" event={"ID":"04fadd0a-3ad7-4392-b989-6ad11dad9432","Type":"ContainerStarted","Data":"938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449"}
Jul 08 03:22:50.990779 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.990611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8" event={"ID":"04fadd0a-3ad7-4392-b989-6ad11dad9432","Type":"ContainerStarted","Data":"1fb373a1ebd734f78111d79b30a74d53c7b083ba6d3c81dac232372ad624be2c"}
Jul 08 03:22:50.990779 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:50.990655    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:22:51.008473 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:51.008419    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8" podStartSLOduration=1.605739427 podStartE2EDuration="2.008406294s" totalImagesPullingTime="402.666867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:22:51.005823122 +0000 UTC m=+2558.933684730" watchObservedRunningTime="2026-07-08 03:22:51.008406294 +0000 UTC m=+2558.936267902"
Jul 08 03:22:57.000969 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:22:57.000938    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:25:12.708755 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:25:12.708634    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:25:12.719387 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:25:12.719368    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzzq2_e4c49f8e-fe3b-4cab-af44-166d9693057c/ovn-acl-logging/0.log"
Jul 08 03:26:06.154037 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.154000    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"]
Jul 08 03:26:06.154525 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.154225    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8" podUID="04fadd0a-3ad7-4392-b989-6ad11dad9432" containerName="maas-api" containerID="cri-o://938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449" gracePeriod=30
Jul 08 03:26:06.397229 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.397207    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:26:06.471059 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.471028    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmj57\" (UniqueName: \"kubernetes.io/projected/04fadd0a-3ad7-4392-b989-6ad11dad9432-kube-api-access-rmj57\") pod \"04fadd0a-3ad7-4392-b989-6ad11dad9432\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") "
Jul 08 03:26:06.471224 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.471067    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04fadd0a-3ad7-4392-b989-6ad11dad9432-openshift-service-ca\") pod \"04fadd0a-3ad7-4392-b989-6ad11dad9432\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") "
Jul 08 03:26:06.471224 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.471092    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls\") pod \"04fadd0a-3ad7-4392-b989-6ad11dad9432\" (UID: \"04fadd0a-3ad7-4392-b989-6ad11dad9432\") "
Jul 08 03:26:06.471403 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.471380    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04fadd0a-3ad7-4392-b989-6ad11dad9432-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04fadd0a-3ad7-4392-b989-6ad11dad9432" (UID: "04fadd0a-3ad7-4392-b989-6ad11dad9432"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:26:06.473151 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.473125    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04fadd0a-3ad7-4392-b989-6ad11dad9432" (UID: "04fadd0a-3ad7-4392-b989-6ad11dad9432"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:26:06.473212 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.473163    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04fadd0a-3ad7-4392-b989-6ad11dad9432-kube-api-access-rmj57" (OuterVolumeSpecName: "kube-api-access-rmj57") pod "04fadd0a-3ad7-4392-b989-6ad11dad9432" (UID: "04fadd0a-3ad7-4392-b989-6ad11dad9432"). InnerVolumeSpecName "kube-api-access-rmj57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:26:06.571634 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.571603    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmj57\" (UniqueName: \"kubernetes.io/projected/04fadd0a-3ad7-4392-b989-6ad11dad9432-kube-api-access-rmj57\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:26:06.571634 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.571634    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04fadd0a-3ad7-4392-b989-6ad11dad9432-openshift-service-ca\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:26:06.571833 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.571644    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04fadd0a-3ad7-4392-b989-6ad11dad9432-maas-api-tls\") on node \"ip-10-0-129-88.ec2.internal\" DevicePath \"\""
Jul 08 03:26:06.908524 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.908436    2558 generic.go:356] "Generic (PLEG): container finished" podID="04fadd0a-3ad7-4392-b989-6ad11dad9432" containerID="938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449" exitCode=0
Jul 08 03:26:06.908524 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.908495    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"
Jul 08 03:26:06.908723 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.908516    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8" event={"ID":"04fadd0a-3ad7-4392-b989-6ad11dad9432","Type":"ContainerDied","Data":"938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449"}
Jul 08 03:26:06.908723 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.908589    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8" event={"ID":"04fadd0a-3ad7-4392-b989-6ad11dad9432","Type":"ContainerDied","Data":"1fb373a1ebd734f78111d79b30a74d53c7b083ba6d3c81dac232372ad624be2c"}
Jul 08 03:26:06.908723 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.908617    2558 scope.go:117] "RemoveContainer" containerID="938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449"
Jul 08 03:26:06.917322 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.917305    2558 scope.go:117] "RemoveContainer" containerID="938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449"
Jul 08 03:26:06.917558 ip-10-0-129-88 kubenswrapper[2558]: E0708 03:26:06.917539    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449\": container with ID starting with 938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449 not found: ID does not exist" containerID="938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449"
Jul 08 03:26:06.917608 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.917566    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449"} err="failed to get container status \"938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449\": rpc error: code = NotFound desc = could not find container \"938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449\": container with ID starting with 938ab2b0296bb45da09d8dc119c6f6a3d93fd35ee8b735f4ea9c67db04e88449 not found: ID does not exist"
Jul 08 03:26:06.925334 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.925315    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"]
Jul 08 03:26:06.929305 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:06.929285    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6493fa-9b956449-p72g8"]
Jul 08 03:26:08.619478 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:08.619444    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04fadd0a-3ad7-4392-b989-6ad11dad9432" path="/var/lib/kubelet/pods/04fadd0a-3ad7-4392-b989-6ad11dad9432/volumes"
Jul 08 03:26:12.806389 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:12.806362    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dd5b94576-lxxn6_2ae60fe1-3884-47f6-907a-f9f550efdf93/authorino/0.log"
Jul 08 03:26:18.260260 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:18.260230    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fbb74d9dd-w285s_bc676c0b-06d2-4c50-b9cc-075164db6a0f/copy-manifests/0.log"
Jul 08 03:26:18.267081 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:18.267062    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fbb74d9dd-w285s_bc676c0b-06d2-4c50-b9cc-075164db6a0f/manager/0.log"
Jul 08 03:26:19.033942 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.033910    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75_8deb1abe-426b-4c70-86e4-1b74b508b12e/util/0.log"
Jul 08 03:26:19.040344 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.040319    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75_8deb1abe-426b-4c70-86e4-1b74b508b12e/pull/0.log"
Jul 08 03:26:19.046532 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.046512    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75_8deb1abe-426b-4c70-86e4-1b74b508b12e/extract/0.log"
Jul 08 03:26:19.163253 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.163232    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6_d5fc2480-3c0d-4926-a2ab-f402e31dc2a0/util/0.log"
Jul 08 03:26:19.170048 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.170033    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6_d5fc2480-3c0d-4926-a2ab-f402e31dc2a0/pull/0.log"
Jul 08 03:26:19.176611 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.176597    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6_d5fc2480-3c0d-4926-a2ab-f402e31dc2a0/extract/0.log"
Jul 08 03:26:19.289359 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.289289    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn_4a98cf78-aa55-4048-bdad-f35e1827893e/util/0.log"
Jul 08 03:26:19.295728 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.295708    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn_4a98cf78-aa55-4048-bdad-f35e1827893e/pull/0.log"
Jul 08 03:26:19.302325 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.302304    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn_4a98cf78-aa55-4048-bdad-f35e1827893e/extract/0.log"
Jul 08 03:26:19.413232 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.413205    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv_e5d3c4a0-dd94-454d-bf03-f82ff578ce24/util/0.log"
Jul 08 03:26:19.419920 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.419899    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv_e5d3c4a0-dd94-454d-bf03-f82ff578ce24/pull/0.log"
Jul 08 03:26:19.426629 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.426612    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv_e5d3c4a0-dd94-454d-bf03-f82ff578ce24/extract/0.log"
Jul 08 03:26:19.542031 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.541953    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dd5b94576-lxxn6_2ae60fe1-3884-47f6-907a-f9f550efdf93/authorino/0.log"
Jul 08 03:26:19.883626 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:19.883548    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mlqxn_b6178861-a443-4fd3-bd5f-213be928d672/kuadrant-console-plugin/0.log"
Jul 08 03:26:20.970811 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:20.970781    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-dcfc578f7-7vslg_7311b400-ba9c-45bc-afed-b19be4974ca6/kube-auth-proxy/0.log"
Jul 08 03:26:21.897579 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:21.897550    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-s42b2_8a4ad3c3-f51b-4b67-b4a1-0d4d7ae0deea/main/0.log"
Jul 08 03:26:22.130665 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:22.130637    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-m97xr_7e2b5d2e-bc56-4578-9a7a-3d01411cb708/main/0.log"
Jul 08 03:26:22.375853 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:22.375810    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2k6bz_f2b6d27e-fe48-418a-a8b8-6ae59aa8af63/main/0.log"
Jul 08 03:26:22.489540 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:22.489505    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pn7tt_83ebbeaf-8cc2-4477-b438-4f51621d69c0/main/0.log"
Jul 08 03:26:30.007346 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:30.007317    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rvgzh_10e31942-6737-4b44-a21c-3009611e2f37/global-pull-secret-syncer/0.log"
Jul 08 03:26:30.239045 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:30.239016    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wg2vj_6f6f7f3c-3e93-4a58-94f0-16e462e437e5/konnectivity-agent/0.log"
Jul 08 03:26:30.306382 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:30.306307    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-88.ec2.internal_1fa326e2985d0d87989ce38a6f8caaeb/haproxy/0.log"
Jul 08 03:26:34.461510 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.461426    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75_8deb1abe-426b-4c70-86e4-1b74b508b12e/extract/0.log"
Jul 08 03:26:34.488523 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.488500    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75_8deb1abe-426b-4c70-86e4-1b74b508b12e/util/0.log"
Jul 08 03:26:34.514022 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.513998    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmj75_8deb1abe-426b-4c70-86e4-1b74b508b12e/pull/0.log"
Jul 08 03:26:34.564291 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.564270    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6_d5fc2480-3c0d-4926-a2ab-f402e31dc2a0/extract/0.log"
Jul 08 03:26:34.591384 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.591364    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6_d5fc2480-3c0d-4926-a2ab-f402e31dc2a0/util/0.log"
Jul 08 03:26:34.622994 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.622969    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kdxj6_d5fc2480-3c0d-4926-a2ab-f402e31dc2a0/pull/0.log"
Jul 08 03:26:34.655437 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.655421    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn_4a98cf78-aa55-4048-bdad-f35e1827893e/extract/0.log"
Jul 08 03:26:34.687681 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.687662    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn_4a98cf78-aa55-4048-bdad-f35e1827893e/util/0.log"
Jul 08 03:26:34.720147 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.720093    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nnftn_4a98cf78-aa55-4048-bdad-f35e1827893e/pull/0.log"
Jul 08 03:26:34.749808 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.749791    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv_e5d3c4a0-dd94-454d-bf03-f82ff578ce24/extract/0.log"
Jul 08 03:26:34.778556 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.778536    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv_e5d3c4a0-dd94-454d-bf03-f82ff578ce24/util/0.log"
Jul 08 03:26:34.807002 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:34.806985    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xmslv_e5d3c4a0-dd94-454d-bf03-f82ff578ce24/pull/0.log"
Jul 08 03:26:35.083458 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:35.083375    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dd5b94576-lxxn6_2ae60fe1-3884-47f6-907a-f9f550efdf93/authorino/0.log"
Jul 08 03:26:35.216007 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:35.215982    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mlqxn_b6178861-a443-4fd3-bd5f-213be928d672/kuadrant-console-plugin/0.log"
Jul 08 03:26:37.970700 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:37.970669    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxnl7_17882e9b-f79a-4875-83f3-46a2985ecc79/node-exporter/0.log"
Jul 08 03:26:37.999277 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:37.999255    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxnl7_17882e9b-f79a-4875-83f3-46a2985ecc79/kube-rbac-proxy/0.log"
Jul 08 03:26:38.030238 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:38.030212    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxnl7_17882e9b-f79a-4875-83f3-46a2985ecc79/init-textfile/0.log"
Jul 08 03:26:39.959628 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.959588    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"]
Jul 08 03:26:39.960106 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.960027    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fadd0a-3ad7-4392-b989-6ad11dad9432" containerName="maas-api"
Jul 08 03:26:39.960106 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.960045    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fadd0a-3ad7-4392-b989-6ad11dad9432" containerName="maas-api"
Jul 08 03:26:39.960217 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.960148    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="04fadd0a-3ad7-4392-b989-6ad11dad9432" containerName="maas-api"
Jul 08 03:26:39.963139 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.963117    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:39.965851 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.965819    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wbm7l\"/\"default-dockercfg-q7m95\""
Jul 08 03:26:39.965981 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.965916    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wbm7l\"/\"openshift-service-ca.crt\""
Jul 08 03:26:39.967136 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.967120    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wbm7l\"/\"kube-root-ca.crt\""
Jul 08 03:26:39.971231 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:39.971211    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"]
Jul 08 03:26:40.026722 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.026694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zzmf\" (UniqueName: \"kubernetes.io/projected/7d6b5142-7977-45a3-9320-0a5f4160f656-kube-api-access-6zzmf\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.026869 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.026728    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-sys\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.026869 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.026753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-proc\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.026869 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.026778    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-lib-modules\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.026869 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.026799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-podres\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127293 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-proc\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127434 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-lib-modules\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127434 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127314    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-podres\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127434 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zzmf\" (UniqueName: \"kubernetes.io/projected/7d6b5142-7977-45a3-9320-0a5f4160f656-kube-api-access-6zzmf\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127434 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-sys\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127434 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127390    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-proc\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127434 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127437    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-sys\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127660 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127437    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-lib-modules\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.127660 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.127457    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7d6b5142-7977-45a3-9320-0a5f4160f656-podres\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.135617 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.135593    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zzmf\" (UniqueName: \"kubernetes.io/projected/7d6b5142-7977-45a3-9320-0a5f4160f656-kube-api-access-6zzmf\") pod \"perf-node-gather-daemonset-mgrwb\" (UID: \"7d6b5142-7977-45a3-9320-0a5f4160f656\") " pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.273551 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.273477    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:40.403750 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:40.403714    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"]
Jul 08 03:26:40.406923 ip-10-0-129-88 kubenswrapper[2558]: W0708 03:26:40.406879    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7d6b5142_7977_45a3_9320_0a5f4160f656.slice/crio-f3d138fac71167a36a33093f485bd6f582224d65b1c51fcf904ce28c26c6adfd WatchSource:0}: Error finding container f3d138fac71167a36a33093f485bd6f582224d65b1c51fcf904ce28c26c6adfd: Status 404 returned error can't find the container with id f3d138fac71167a36a33093f485bd6f582224d65b1c51fcf904ce28c26c6adfd
Jul 08 03:26:41.066075 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:41.066037    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb" event={"ID":"7d6b5142-7977-45a3-9320-0a5f4160f656","Type":"ContainerStarted","Data":"83da6702a471f6f792511ee7aada2e726b8cac5265f77f8bc652cfbf06df2b57"}
Jul 08 03:26:41.066440 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:41.066090    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb" event={"ID":"7d6b5142-7977-45a3-9320-0a5f4160f656","Type":"ContainerStarted","Data":"f3d138fac71167a36a33093f485bd6f582224d65b1c51fcf904ce28c26c6adfd"}
Jul 08 03:26:41.066440 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:41.066182    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:41.082627 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:41.082584    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb" podStartSLOduration=2.082571076 podStartE2EDuration="2.082571076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:26:41.081268377 +0000 UTC m=+2789.009129986" watchObservedRunningTime="2026-07-08 03:26:41.082571076 +0000 UTC m=+2789.010432684"
Jul 08 03:26:41.122112 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:41.122088    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d7bf7d9f7-zswmz_d4ca526e-6dfe-4f1c-a9b5-27c32fdde440/console/0.log"
Jul 08 03:26:42.841824 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:42.841800    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8hwx_d011ddb5-0557-4f54-b4a5-22abc3e9798c/dns/0.log"
Jul 08 03:26:42.865994 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:42.865975    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8hwx_d011ddb5-0557-4f54-b4a5-22abc3e9798c/kube-rbac-proxy/0.log"
Jul 08 03:26:43.023507 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:43.023476    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p259t_b4647bc1-61bf-4ef6-9547-559ff6f3ec49/dns-node-resolver/0.log"
Jul 08 03:26:43.618614 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:43.618585    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b74d44cc5-n5rrr_14b3f5ab-1cb9-45c6-a468-8f539636528e/registry/0.log"
Jul 08 03:26:43.626379 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:43.626360    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b74d44cc5-n5rrr_14b3f5ab-1cb9-45c6-a468-8f539636528e/registry/1.log"
Jul 08 03:26:43.710268 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:43.710243    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tts6l_30e3fc41-317a-4aa5-aa4c-c76011d1d220/node-ca/0.log"
Jul 08 03:26:45.019368 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:45.019333    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-dcfc578f7-7vslg_7311b400-ba9c-45bc-afed-b19be4974ca6/kube-auth-proxy/0.log"
Jul 08 03:26:45.998704 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:45.998672    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrmmk_72c48082-e39b-4908-a3b9-07cf0759e80b/serve-healthcheck-canary/0.log"
Jul 08 03:26:46.704090 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:46.704062    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jl2lq_d16de454-b9dc-4872-a160-26e4e0db8a20/kube-rbac-proxy/0.log"
Jul 08 03:26:46.726677 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:46.726649    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jl2lq_d16de454-b9dc-4872-a160-26e4e0db8a20/exporter/0.log"
Jul 08 03:26:46.750538 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:46.750519    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jl2lq_d16de454-b9dc-4872-a160-26e4e0db8a20/extractor/0.log"
Jul 08 03:26:47.080252 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:47.080183    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wbm7l/perf-node-gather-daemonset-mgrwb"
Jul 08 03:26:49.835344 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:49.835310    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fbb74d9dd-w285s_bc676c0b-06d2-4c50-b9cc-075164db6a0f/manager/0.log"
Jul 08 03:26:49.859805 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:49.859778    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fbb74d9dd-w285s_bc676c0b-06d2-4c50-b9cc-075164db6a0f/copy-manifests/0.log"
Jul 08 03:26:51.138041 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:51.138012    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56954ff8c6-khqvt_9b189f01-3984-4478-a7cf-9aa1e0d36a43/manager/0.log"
Jul 08 03:26:51.406851 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:51.406809    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-dcfc578f7-7vslg_7311b400-ba9c-45bc-afed-b19be4974ca6/kube-auth-proxy/0.log"
Jul 08 03:26:53.007421 ip-10-0-129-88 kubenswrapper[2558]: I0708 03:26:53.007388    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-dcfc578f7-7vslg_7311b400-ba9c-45bc-afed-b19be4974ca6/kube-auth-proxy/0.log"