Jul 17 05:32:55.015014 ip-10-0-128-189 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 05:32:55.535748 ip-10-0-128-189 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 05:32:55.535748 ip-10-0-128-189 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 05:32:55.535748 ip-10-0-128-189 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 05:32:55.535748 ip-10-0-128-189 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 05:32:55.535748 ip-10-0-128-189 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 05:32:55.536946 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.536772    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 05:32:55.541603 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541581    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:32:55.541603 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541602    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541606    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541610    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541613    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541616    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541619    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541622    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541625    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541627    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541630    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541632    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541635    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541638    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541641    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541643    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541646    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541648    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541651    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541659    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:32:55.541680 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541662    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541666    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541669    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541672    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541674    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541677    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541679    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541682    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541685    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541688    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541691    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541695    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541699    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541702    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541705    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541709    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541712    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541715    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541718    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:32:55.542158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541721    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541724    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541727    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541730    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541733    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541736    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541738    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541741    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541743    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541746    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541748    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541752    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541762    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541765    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541768    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541772    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541774    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541777    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541780    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541783    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:32:55.542627 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541786    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541789    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541791    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541794    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541797    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541799    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541802    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541804    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541807    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541809    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541812    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541815    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541817    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541820    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541824    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541828    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541831    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541834    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541836    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541839    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:32:55.543234 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541842    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541845    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541849    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541852    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541871    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541874    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541877    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541880    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541883    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541887    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541890    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541893    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541896    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541899    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541902    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541904    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541907    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541910    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541912    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:32:55.543751 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.541915    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542628    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542634    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542637    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542640    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542644    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542646    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542651    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542655    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542658    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542661    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542664    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542667    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542669    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542672    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542675    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542678    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542681    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542683    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542687    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:32:55.544249 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542690    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542693    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542696    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542698    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542701    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542703    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542706    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542709    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542711    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542714    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542716    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542719    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542721    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542724    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542726    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542730    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542733    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542735    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542739    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542742    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:32:55.544769 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542745    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542748    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542751    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542753    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542756    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542759    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542761    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542764    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542767    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542770    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542773    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542775    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542778    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542781    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542784    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542786    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542789    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542792    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542794    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542797    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:32:55.545312 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542800    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542802    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542805    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542807    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542810    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542812    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542818    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542820    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542823    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542826    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542829    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542832    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542834    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542836    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542839    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542841    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542844    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542847    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542849    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:32:55.545830 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542852    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542872    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542877    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542879    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542882    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542885    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542888    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542891    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542893    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542896    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542899    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542901    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542904    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542906    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542911    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542914    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542917    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542920    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542923    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:32:55.546340 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542926    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.542929    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543312    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543317    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543320    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543324    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543328    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543331    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543334    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543337    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543340    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543343    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543346    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543349    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543353    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543356    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543359    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543361    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543364    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543367    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:32:55.546848 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543369    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543372    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543374    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543376    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543379    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543381    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543384    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543387    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543389    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543392    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543394    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543397    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543402    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543404    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543406    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543409    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543412    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543416    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543419    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:32:55.547364 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543423    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543427    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543430    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543432    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543435    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543438    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543440    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543444    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543447    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543450    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543452    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543455    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543457    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543460    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543463    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543465    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543468    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543471    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543473    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543475    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:32:55.547836 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543478    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543480    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543483    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543486    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543488    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543492    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543494    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543497    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543499    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543502    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543505    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543507    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543510    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543512    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543515    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543518    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543521    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543523    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543525    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543529    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:32:55.548390 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543532    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543534    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543537    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543540    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543543    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543545    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543548    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543551    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543553    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543556    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543558    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543561    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543563    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543566    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543568    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543571    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543574    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543578    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543580    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543583    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:32:55.548924 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543585    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.543588    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543675    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543687    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543693    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543698    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543702    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543705    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543709    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543713    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543717    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543721    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543724    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543727    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543731    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543734    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543737    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543740    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543743    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543746    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543751    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543754    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543758    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543761    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 05:32:55.549423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543764    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543768    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543771    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543774    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543778    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543783    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543786    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543789    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543792    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543795    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543800    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543802    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543806    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543809    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543811    2559 flags.go:64] FLAG: --enable-server="true"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543814    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543819    2559 flags.go:64] FLAG: --event-burst="100"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543822    2559 flags.go:64] FLAG: --event-qps="50"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543825    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543829    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543832    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543836    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543838    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543841    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543844    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 17 05:32:55.550060 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543847    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543850    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543868    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543872    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543875    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543878    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543881    2559 flags.go:64] FLAG: --feature-gates=""
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543885    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543888    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543891    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543894    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543897    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543902    2559 flags.go:64] FLAG: --help="false"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543905    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543908    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543911    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543914    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543917    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543921    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543924    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543927    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543930    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543933    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 05:32:55.550659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543935    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543939    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543942    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543945    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543948    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543951    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543954    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543957    2559 flags.go:64] FLAG: --lock-file=""
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543960    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543962    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543965    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543971    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543974    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543977    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543980    2559 flags.go:64] FLAG: --logging-format="text"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543983    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543987    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543989    2559 flags.go:64] FLAG: --manifest-url=""
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543992    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543996    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.543999    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544008    2559 flags.go:64] FLAG: --max-pods="110"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544011    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544014    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544017    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 05:32:55.551268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544020    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544024    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544027    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544030    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544038    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544041    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544044    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544047    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544050    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544057    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544060    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544064    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544067    2559 flags.go:64] FLAG: --port="10250"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544070    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544073    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-021e1eadd2b858643"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544077    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544079    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544083    2559 flags.go:64] FLAG: --register-node="true"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544086    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544090    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544093    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544096    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544099    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544103    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544106    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 05:32:55.552004 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544109    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544112    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544115    2559 flags.go:64] FLAG: --runonce="false"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544119    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544123    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544125    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544128    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544131    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544135    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544138    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544141    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544144    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544148    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544150    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544153    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544156    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544159    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544162    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544168    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544171    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544174    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544178    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544181    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544184    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544186    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 05:32:55.553032 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544189    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544193    2559 flags.go:64] FLAG: --v="2"
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544197    2559 flags.go:64] FLAG: --version="false"
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544201    2559 flags.go:64] FLAG: --vmodule=""
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544205    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.544209    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544331    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544335    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544338    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544341    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544345    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544348    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544350    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544353    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544356    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544358    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544361    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544363    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544366    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544368    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544371    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:32:55.553765 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544374    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544377    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544380    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544382    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544385    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544387    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544390    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544393    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544395    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544398    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544401    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544403    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544406    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544408    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544411    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544414    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544417    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544420    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544422    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544425    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:32:55.554321 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544428    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544432    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544435    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544437    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544440    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544443    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544445    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544448    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544450    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544453    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544456    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544459    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544463    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544466    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544469    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544471    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544474    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544477    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544479    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544482    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:32:55.554823 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544485    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544488    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544490    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544493    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544496    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544499    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544502    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544504    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544507    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544510    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544513    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544516    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544520    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544525    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544528    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544531    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544536    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544539    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544542    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:32:55.555346 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544545    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544548    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544550    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544553    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544555    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544560    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544562    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544565    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544568    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544571    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544573    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544576    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544579    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544581    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544584    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544586    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544590    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544594    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544597    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:32:55.555835 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544600    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544603    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544606    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544608    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544612    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.544614    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.545440    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.554636    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.554659    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.554682    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.554688    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554774    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554779    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554783    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554786    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:32:55.556330 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554789    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554791    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554795    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554797    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554801    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554804    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554807    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554809    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554812    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554815    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554818    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554822    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554827    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554831    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554834    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554838    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554841    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554844    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:32:55.556713 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554847    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554850    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554868    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554874    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554877    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554881    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554883    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554886    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554889    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554892    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554895    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554898    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554900    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554903    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554906    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554908    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554911    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554914    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554917    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554920    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:32:55.557194 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554923    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554925    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554928    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554931    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554934    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554937    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554939    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554942    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554944    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554947    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554950    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554952    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554955    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554957    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554960    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554963    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554965    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554968    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554971    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554974    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:32:55.557688 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554976    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554979    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554981    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554984    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554987    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554989    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554992    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554995    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.554997    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555000    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555003    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555005    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555008    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555011    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555013    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555016    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555019    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555023    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555025    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555028    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:32:55.558228 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555031    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555033    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555036    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555039    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555041    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555044    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555046    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555049    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555052    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555055    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555057    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555060    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555062    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555065    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555068    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555070    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:32:55.558735 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555074    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.555080    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555218    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555223    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555226    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555229    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555232    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555234    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555237    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555240    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555243    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555246    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555249    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555251    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555255    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:32:55.559158 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555258    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555260    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555263    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555265    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555268    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555271    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555273    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555276    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555278    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555281    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555284    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555286    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555289    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555291    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555294    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555296    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555299    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555301    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555304    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555307    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:32:55.559532 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555309    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555312    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555315    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555317    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555320    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555323    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555326    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555329    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555332    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555335    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555338    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555341    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555344    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555347    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555349    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555352    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555355    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555358    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555360    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555363    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:32:55.560036 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555366    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555370    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555373    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555376    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555380    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555383    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555385    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555389    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555391    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555394    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555397    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555399    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555402    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555404    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555407    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555409    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555412    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555415    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555418    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:32:55.560565 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555420    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555422    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555425    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555428    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555430    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555433    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555436    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555439    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555443    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555446    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555449    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555452    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555455    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555458    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555461    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555464    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555467    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555470    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555472    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:32:55.561116 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555474    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555477    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555479    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555482    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555484    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555487    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555492    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:55.555495    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.555500    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.555635    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 05:32:55.561597 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.560159    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 05:32:55.561915 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.561760    2559 server.go:1019] "Starting client certificate rotation"
Jul 17 05:32:55.561966 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.561901    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 05:32:55.561966 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.561957    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 05:32:55.564722 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.564699    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 05:32:55.569793 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.569769    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 05:32:55.581913 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.581885    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 17 05:32:55.584586 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.584563    2559 log.go:25] "Validated CRI v1 image API"
Jul 17 05:32:55.585421 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.585401    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 05:32:55.588223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.588199    2559 fs.go:135] Filesystem UUIDs: map[526af883-7cb9-45b5-a87b-f6fcdacf940b:/dev/nvme0n1p4 6a24f493-0ab8-45d0-a5fb-cc3f78018896:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 17 05:32:55.588290 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.588222    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 05:32:55.590868 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.590837    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 05:32:55.594629 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.594488    2559 manager.go:216] Machine: {Timestamp:2026-07-17 05:32:55.592619425 +0000 UTC m=+0.371102365 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100549 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29e4fa100e6c9c9df0302edaa03f3a SystemUUID:ec29e4fa-100e-6c9c-9df0-302edaa03f3a BootID:40c9dcd0-1a84-4821-9c9a-0f3fc1d6cadf Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b8:d1:b2:c5:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b8:d1:b2:c5:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:0b:5d:7a:14:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 05:32:55.594629 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.594610    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 05:32:55.594780 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.594708    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 05:32:55.598685 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.598648    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 05:32:55.598843 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.598689    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-189.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 05:32:55.599594 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.599582    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 05:32:55.599633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.599597    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 05:32:55.599633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.599605    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 05:32:55.600377 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.600364    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 05:32:55.602257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.602240    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 05:32:55.605868 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.605843    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 05:32:55.606012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606002    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 05:32:55.606057 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606037    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 05:32:55.606057 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606048    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 05:32:55.606126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606061    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 05:32:55.606126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606072    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 17 05:32:55.606126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606082    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 05:32:55.606516 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606501    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 05:32:55.606578 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.606524    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 05:32:55.609019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.608999    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 05:32:55.611500 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.611478    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 05:32:55.611599 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.611512    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 05:32:55.612883 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612870    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 05:32:55.612924 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612888    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 05:32:55.612924 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612894    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 05:32:55.612924 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612900    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 05:32:55.612924 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612909    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 05:32:55.612924 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612915    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 05:32:55.612924 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612923    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 05:32:55.613125 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612929    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 05:32:55.613125 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612936    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 05:32:55.613125 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612942    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 05:32:55.613125 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612952    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 05:32:55.613125 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.612957    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 05:32:55.614700 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.614687    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 05:32:55.614743 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.614702    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 05:32:55.618315 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.618301    2559 server.go:1268] "Started kubelet"
Jul 17 05:32:55.618455 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.618383    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 05:32:55.618695 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.618395    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 05:32:55.618695 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.618590    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 05:32:55.619201 ip-10-0-128-189 systemd[1]: Started Kubernetes Kubelet.
Jul 17 05:32:55.620166 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.619921    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 05:32:55.620166 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.619964    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 05:32:55.620166 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.620022    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-189.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 05:32:55.620166 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.619913    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 05:32:55.620945 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.620927    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 05:32:55.624475 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.623373    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-189.ec2.internal.18c2fcd0af668752  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-189.ec2.internal,UID:ip-10-0-128-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-189.ec2.internal,},FirstTimestamp:2026-07-17 05:32:55.618266962 +0000 UTC m=+0.396749964,LastTimestamp:2026-07-17 05:32:55.618266962 +0000 UTC m=+0.396749964,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-189.ec2.internal,}"
Jul 17 05:32:55.624765 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.624749    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 05:32:55.630147 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.630127    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 05:32:55.630232 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.630132    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 05:32:55.630778 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.630637    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:55.631099 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631052    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 05:32:55.631099 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631072    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 05:32:55.631253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631176    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 05:32:55.631253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631187    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 05:32:55.631253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631185    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 05:32:55.631693 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631677    2559 factory.go:55] Registering systemd factory
Jul 17 05:32:55.631772 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631717    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 17 05:32:55.632000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.631985    2559 factory.go:153] Registering CRI-O factory
Jul 17 05:32:55.632044 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.632004    2559 factory.go:223] Registration of the crio container factory successfully
Jul 17 05:32:55.632091 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.632057    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 05:32:55.632091 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.632084    2559 factory.go:103] Registering Raw factory
Jul 17 05:32:55.632163 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.632106    2559 manager.go:1196] Started watching for new ooms in manager
Jul 17 05:32:55.633533 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.633509    2559 manager.go:349] Starting recovery of all containers
Jul 17 05:32:55.639037 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.638985    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dw4kl"
Jul 17 05:32:55.642444 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.642407    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 05:32:55.642545 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.642505    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 05:32:55.643907 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.643885    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dw4kl"
Jul 17 05:32:55.646601 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.646585    2559 manager.go:354] Recovery completed
Jul 17 05:32:55.651007 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.650993    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.653440 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.653419    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.653523 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.653455    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.653523 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.653466    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.654065 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.654049    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 05:32:55.654065 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.654063    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 05:32:55.654185 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.654080    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 05:32:55.654185 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.654063    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.655145 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655130    2559 policy_none.go:49] "None policy: Start"
Jul 17 05:32:55.655145 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655147    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 05:32:55.655257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655157    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 05:32:55.655535 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655515    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.655629 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655556    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.655629 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655570    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.655978 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.655965    2559 policy_none.go:47] "Start"
Jul 17 05:32:55.691879 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.691619    2559 manager.go:337] "Starting Device Plugin manager"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.691926    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.691942    2559 server.go:87] "Starting device plugin registration server"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.692341    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.692357    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.692451    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.693052    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.693063    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.693711    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 05:32:55.706781 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.693767    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:55.744180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.744140    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 05:32:55.745366 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.745333    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 05:32:55.745475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.745387    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 05:32:55.745475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.745425    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 05:32:55.745475 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.745468    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 05:32:55.748079 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.748053    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:32:55.793220 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.793142    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.794230 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.794213    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.794305 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.794245    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.794305 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.794258    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.794367 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.794306    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.805310 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.805288    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.805437 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.805321    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-189.ec2.internal\": node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:55.829239 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.829209    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:55.845967 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.845934    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"]
Jul 17 05:32:55.846079 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.846025    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.846999 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.846983    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.847063 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.847014    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.847063 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.847024    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.848915 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.848901    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.849035 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849018    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.849081 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849053    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.849871 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849840    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.849871 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849856    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.850000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849881    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.850000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849881    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.850000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849911    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.850000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.849897    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.851922 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.851902    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.852034 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.851937    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:32:55.853003 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.852981    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:32:55.853093 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.853013    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:32:55.853093 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.853029    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:32:55.878041 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.878004    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-189.ec2.internal\" not found" node="ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.883242 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.883219    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-189.ec2.internal\" not found" node="ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.930352 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:55.930307    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:55.932745 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.932723    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/306dc5b5dbae4709c425668aaebe9719-config\") pod \"kube-apiserver-proxy-ip-10-0-128-189.ec2.internal\" (UID: \"306dc5b5dbae4709c425668aaebe9719\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.932815 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.932754    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75da3d20d9ebca413dcb11475cc9a3b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal\" (UID: \"75da3d20d9ebca413dcb11475cc9a3b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:55.932815 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:55.932775    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75da3d20d9ebca413dcb11475cc9a3b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal\" (UID: \"75da3d20d9ebca413dcb11475cc9a3b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.031146 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.031081    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:56.033536 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.033509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/306dc5b5dbae4709c425668aaebe9719-config\") pod \"kube-apiserver-proxy-ip-10-0-128-189.ec2.internal\" (UID: \"306dc5b5dbae4709c425668aaebe9719\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.033622 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.033545    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75da3d20d9ebca413dcb11475cc9a3b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal\" (UID: \"75da3d20d9ebca413dcb11475cc9a3b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.033622 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.033565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75da3d20d9ebca413dcb11475cc9a3b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal\" (UID: \"75da3d20d9ebca413dcb11475cc9a3b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.033697 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.033621    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75da3d20d9ebca413dcb11475cc9a3b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal\" (UID: \"75da3d20d9ebca413dcb11475cc9a3b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.033697 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.033628    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75da3d20d9ebca413dcb11475cc9a3b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal\" (UID: \"75da3d20d9ebca413dcb11475cc9a3b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.033697 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.033622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/306dc5b5dbae4709c425668aaebe9719-config\") pod \"kube-apiserver-proxy-ip-10-0-128-189.ec2.internal\" (UID: \"306dc5b5dbae4709c425668aaebe9719\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.131928 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.131788    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:56.184144 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.184114    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.186124 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.186107    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.232437 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.232403    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:56.334020 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.333957    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:56.435273 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.435159    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-189.ec2.internal\" not found"
Jul 17 05:32:56.499128 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.499092    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:32:56.532997 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.532962    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.549694 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.549661    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 05:32:56.551444 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.551426    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal"
Jul 17 05:32:56.559028 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.558999    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 05:32:56.561224 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.561201    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 05:32:56.561404 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.561376    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 05:32:56.591264 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.591241    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:32:56.606502 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.606466    2559 apiserver.go:52] "Watching apiserver"
Jul 17 05:32:56.616136 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.616109    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 05:32:56.616525 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.616502    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-4l8jz","kube-system/global-pull-secret-syncer-7gz6h","kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn","openshift-image-registry/node-ca-47vtj","openshift-multus/multus-additional-cni-plugins-c2f5t","openshift-multus/multus-fb5fx","openshift-multus/network-metrics-daemon-rk2tl","openshift-network-diagnostics/network-check-target-mt2jm","kube-system/konnectivity-agent-mhjrt","openshift-cluster-node-tuning-operator/tuned-ml2m5","openshift-dns/node-resolver-4mb5k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal","openshift-network-operator/iptables-alerter-f242z"]
Jul 17 05:32:56.619380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.619355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.621997 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.621964    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.622151 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.622060    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:32:56.622151 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622100    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 05:32:56.622289 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622145    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 05:32:56.622289 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622152    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.622427 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622298    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nhf84\""
Jul 17 05:32:56.622427 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622420    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.622554 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622428    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 05:32:56.622679 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.622661    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 05:32:56.624700 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.624678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.627052 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.627033    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dl297\""
Jul 17 05:32:56.627324 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.627304    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.627561 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.627541    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.627673 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.627618    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 05:32:56.627673 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.627627    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.629850 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.629833    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.629947 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.629880    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vthv2\""
Jul 17 05:32:56.630153 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.630135    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 05:32:56.630306 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.630290    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 05:32:56.630449 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.630433    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.630559 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.630542    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.632804 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.632787    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 05:32:56.633918 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.633270    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 05:32:56.633918 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.633359    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 05:32:56.633918 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.633539    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.633918 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.633674    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7kv7b\""
Jul 17 05:32:56.633918 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.633798    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.636074 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.636044    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637073 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0833fd9c-124f-4b64-9618-cac8678daad0-serviceca\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.637163 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637082    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-k8s-cni-cncf-io\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637163 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637099    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-cni-multus\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637163 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-slash\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.637163 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-cni-bin\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.637301 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-env-overrides\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.637301 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovnkube-script-lib\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.637301 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637293    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-os-release\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637408 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-cni-bin\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637408 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637331    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-hostroot\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637408 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637347    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovnkube-config\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.637408 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.637408 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637416    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-772sd\" (UniqueName: \"kubernetes.io/projected/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-kube-api-access-772sd\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637456    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7b84b7a-c01d-4f80-8605-b8624b272347-cni-binary-copy\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-socket-dir-parent\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xznlz\" (UniqueName: \"kubernetes.io/projected/2aa96119-a12d-460d-9b53-d2c15e27bc59-kube-api-access-xznlz\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637533    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-285nz\" (UniqueName: \"kubernetes.io/projected/0833fd9c-124f-4b64-9618-cac8678daad0-kube-api-access-285nz\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnxgq\" (UniqueName: \"kubernetes.io/projected/c7b84b7a-c01d-4f80-8605-b8624b272347-kube-api-access-vnxgq\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.637606 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-kubelet-config\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-os-release\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-netns\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-run-ovn-kubernetes\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-system-cni-dir\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-ovn\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637751    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-node-log\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cnibin\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-conf-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637813    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-kubelet\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-registration-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4rj6\" (UniqueName: \"kubernetes.io/projected/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-kube-api-access-d4rj6\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-cnibin\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-socket-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.637995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-etc-selinux\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.638046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638041    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-cni-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638078    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-etc-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-log-socket\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovn-node-metrics-cert\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-systemd-units\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-var-lib-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-sys-fs\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-system-cni-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638223    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-multus-certs\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0833fd9c-124f-4b64-9618-cac8678daad0-host\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-run-netns\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-cni-netd\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638361    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638352    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cni-binary-copy\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638461    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-etc-kubernetes\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.638845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638485    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638440    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pmlv8\""
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-dbus\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-kubelet\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638591    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-daemon-config\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-systemd\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.638647    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-device-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.639277    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:56.639783 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.639348    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:32:56.640601 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.640576    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 05:32:56.642038 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.642009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:32:56.642136 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.642070    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:32:56.644775 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.644757    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.647162 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.647133    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 05:32:56.647297 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.647245    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m5bmz\""
Jul 17 05:32:56.647297 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.647253    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 05:32:56.647412 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.647372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.649282 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.649230    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 05:27:55 +0000 UTC" deadline="2027-12-13 00:41:21.853597159 +0000 UTC"
Jul 17 05:32:56.649282 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.649280    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12331h8m25.204319661s"
Jul 17 05:32:56.649651 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.649633    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n2c5v\""
Jul 17 05:32:56.650034 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.650018    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.650110 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.650024    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.650545 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.650522    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.652633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.652614    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.652747 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.652729    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.652862 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.652847    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-prbkg\""
Jul 17 05:32:56.653374 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.653357    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.655657 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.655638    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 05:32:56.655657 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.655649    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:32:56.655925 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.655807    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 05:32:56.655925 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.655830    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4rb8z\""
Jul 17 05:32:56.690013 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.689911    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:32:56.713166 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.713135    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-csqqx"
Jul 17 05:32:56.721819 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.721794    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-csqqx"
Jul 17 05:32:56.732196 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.732173    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 05:32:56.739996 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.739956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0833fd9c-124f-4b64-9618-cac8678daad0-host\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.739996 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.739996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-run-netns\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740082    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0833fd9c-124f-4b64-9618-cac8678daad0-host\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740149    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-run-netns\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740158    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-cni-netd\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cni-binary-copy\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-cni-netd\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740291    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-etc-kubernetes\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740344    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-dbus\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-kubelet\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-etc-kubernetes\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-daemon-config\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-systemd\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740466    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-device-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0833fd9c-124f-4b64-9618-cac8678daad0-serviceca\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740526    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-k8s-cni-cncf-io\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-cni-multus\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-slash\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.740610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740599    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-cni-bin\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740623    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-env-overrides\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740647    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovnkube-script-lib\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-dbus\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740671    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-os-release\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740696    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-cni-bin\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740716    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-k8s-cni-cncf-io\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-hostroot\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740762    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-cni-multus\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740769    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-cni-bin\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovnkube-config\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-hostroot\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-slash\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-var-lib-kubelet\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740910    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-tuned\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.741423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-cni-bin\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740927    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cni-binary-copy\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8649966-7fcf-46ad-af87-392af43bb3df-hosts-file\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.740981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-772sd\" (UniqueName: \"kubernetes.io/projected/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-kube-api-access-772sd\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7b84b7a-c01d-4f80-8605-b8624b272347-cni-binary-copy\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v95vd\" (UniqueName: \"kubernetes.io/projected/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-kube-api-access-v95vd\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-daemon-config\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8649966-7fcf-46ad-af87-392af43bb3df-tmp-dir\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q7kh\" (UniqueName: \"kubernetes.io/projected/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-kube-api-access-7q7kh\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741143    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-socket-dir-parent\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741145    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-systemd\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-device-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741174    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xznlz\" (UniqueName: \"kubernetes.io/projected/2aa96119-a12d-460d-9b53-d2c15e27bc59-kube-api-access-xznlz\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysctl-conf\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.741115    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovnkube-config\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.742260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-env-overrides\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.741410    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:32:57.241392465 +0000 UTC m=+2.019875409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741434    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdb15470-3a00-4dc9-9b56-ba56e4661ab1-agent-certs\") pod \"konnectivity-agent-mhjrt\" (UID: \"fdb15470-3a00-4dc9-9b56-ba56e4661ab1\") " pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysctl-d\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-host\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-host-slash\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-os-release\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-285nz\" (UniqueName: \"kubernetes.io/projected/0833fd9c-124f-4b64-9618-cac8678daad0-kube-api-access-285nz\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-socket-dir-parent\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnxgq\" (UniqueName: \"kubernetes.io/projected/c7b84b7a-c01d-4f80-8605-b8624b272347-kube-api-access-vnxgq\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-kubelet-config\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-systemd\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742208    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-os-release\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741977    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-kubelet-config\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:56.743253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0833fd9c-124f-4b64-9618-cac8678daad0-serviceca\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-netns\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovnkube-script-lib\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.741973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-os-release\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-run-ovn-kubernetes\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742105    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7b84b7a-c01d-4f80-8605-b8624b272347-cni-binary-copy\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-netns\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-iptables-alerter-script\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-run-ovn-kubernetes\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-system-cni-dir\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-ovn\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-system-cni-dir\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-node-log\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdb15470-3a00-4dc9-9b56-ba56e4661ab1-konnectivity-ca\") pod \"konnectivity-agent-mhjrt\" (UID: \"fdb15470-3a00-4dc9-9b56-ba56e4661ab1\") " pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-node-log\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pztbt\" (UniqueName: \"kubernetes.io/projected/b8649966-7fcf-46ad-af87-392af43bb3df-kube-api-access-pztbt\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.744050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742450    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-run-ovn\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cnibin\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742554    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-cnibin\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-conf-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-kubelet\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742607    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-conf-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742610    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-registration-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742641    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-registration-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4rj6\" (UniqueName: \"kubernetes.io/projected/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-kube-api-access-d4rj6\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742684    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-kubelet\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-modprobe-d\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-cnibin\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742764    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-socket-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-etc-selinux\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-cnibin\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.744699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-kubernetes\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742957    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-cni-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-socket-dir\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-etc-selinux\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.742986    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-etc-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-etc-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-log-socket\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovn-node-metrics-cert\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-multus-cni-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-run\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-log-socket\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743136    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-systemd-units\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-var-lib-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-sys-fs\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743187    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-systemd-units\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2aa96119-a12d-460d-9b53-d2c15e27bc59-var-lib-openvswitch\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.745294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysconfig\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-sys-fs\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-sys\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743300    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-lib-modules\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-var-lib-kubelet\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-tmp\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-system-cni-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-multus-certs\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743421    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743459    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-host-run-multus-certs\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743468    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7b84b7a-c01d-4f80-8605-b8624b272347-system-cni-dir\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.746046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.743461    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl54h\" (UniqueName: \"kubernetes.io/projected/6a08ba58-2146-46da-bb66-1132a32b2419-kube-api-access-dl54h\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:56.746969 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.746910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2aa96119-a12d-460d-9b53-d2c15e27bc59-ovn-node-metrics-cert\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.750689 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.750666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xznlz\" (UniqueName: \"kubernetes.io/projected/2aa96119-a12d-460d-9b53-d2c15e27bc59-kube-api-access-xznlz\") pod \"ovnkube-node-4l8jz\" (UID: \"2aa96119-a12d-460d-9b53-d2c15e27bc59\") " pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.750835 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.750701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnxgq\" (UniqueName: \"kubernetes.io/projected/c7b84b7a-c01d-4f80-8605-b8624b272347-kube-api-access-vnxgq\") pod \"multus-fb5fx\" (UID: \"c7b84b7a-c01d-4f80-8605-b8624b272347\") " pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:56.750973 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.750947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-772sd\" (UniqueName: \"kubernetes.io/projected/3c70031c-daa9-4b4e-a4a4-0c52ac2d0354-kube-api-access-772sd\") pod \"multus-additional-cni-plugins-c2f5t\" (UID: \"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354\") " pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:56.751614 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.751590    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-285nz\" (UniqueName: \"kubernetes.io/projected/0833fd9c-124f-4b64-9618-cac8678daad0-kube-api-access-285nz\") pod \"node-ca-47vtj\" (UID: \"0833fd9c-124f-4b64-9618-cac8678daad0\") " pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.751873 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.751855    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4rj6\" (UniqueName: \"kubernetes.io/projected/d9dba5b8-53a2-4d53-a7b1-d7d54628d546-kube-api-access-d4rj6\") pod \"aws-ebs-csi-driver-node-9m7zn\" (UID: \"d9dba5b8-53a2-4d53-a7b1-d7d54628d546\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.804555 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:56.804508    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306dc5b5dbae4709c425668aaebe9719.slice/crio-a09e791f7b0ee305eacdc6bf366e219607c02c44540453669db5c9aa80b7c4f3 WatchSource:0}: Error finding container a09e791f7b0ee305eacdc6bf366e219607c02c44540453669db5c9aa80b7c4f3: Status 404 returned error can't find the container with id a09e791f7b0ee305eacdc6bf366e219607c02c44540453669db5c9aa80b7c4f3
Jul 17 05:32:56.804824 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:56.804804    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75da3d20d9ebca413dcb11475cc9a3b5.slice/crio-1113dd6d00899a09778d2e0cb7a873aabf0cf912a23f6ed4638462e613574165 WatchSource:0}: Error finding container 1113dd6d00899a09778d2e0cb7a873aabf0cf912a23f6ed4638462e613574165: Status 404 returned error can't find the container with id 1113dd6d00899a09778d2e0cb7a873aabf0cf912a23f6ed4638462e613574165
Jul 17 05:32:56.810569 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.810546    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:32:56.843840 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.843804    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-tuned\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.843840 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.843836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8649966-7fcf-46ad-af87-392af43bb3df-hosts-file\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.844084 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.843864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v95vd\" (UniqueName: \"kubernetes.io/projected/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-kube-api-access-v95vd\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844084 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.843905    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8649966-7fcf-46ad-af87-392af43bb3df-tmp-dir\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.844084 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.843942    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8649966-7fcf-46ad-af87-392af43bb3df-hosts-file\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.844084 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q7kh\" (UniqueName: \"kubernetes.io/projected/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-kube-api-access-7q7kh\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844082    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysctl-conf\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdb15470-3a00-4dc9-9b56-ba56e4661ab1-agent-certs\") pod \"konnectivity-agent-mhjrt\" (UID: \"fdb15470-3a00-4dc9-9b56-ba56e4661ab1\") " pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysctl-d\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-host\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-host-slash\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-systemd\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844268 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8649966-7fcf-46ad-af87-392af43bb3df-tmp-dir\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-systemd\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysctl-conf\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-iptables-alerter-script\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844302    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-host-slash\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844337    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysctl-d\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdb15470-3a00-4dc9-9b56-ba56e4661ab1-konnectivity-ca\") pod \"konnectivity-agent-mhjrt\" (UID: \"fdb15470-3a00-4dc9-9b56-ba56e4661ab1\") " pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pztbt\" (UniqueName: \"kubernetes.io/projected/b8649966-7fcf-46ad-af87-392af43bb3df-kube-api-access-pztbt\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844409    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-modprobe-d\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-kubernetes\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-host\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-run\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysconfig\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844504    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-modprobe-d\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844535    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-sys\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844584    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-lib-modules\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844594    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-run\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.844638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-sysconfig\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-kubernetes\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-var-lib-kubelet\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-sys\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-var-lib-kubelet\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844693    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-tmp\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl54h\" (UniqueName: \"kubernetes.io/projected/6a08ba58-2146-46da-bb66-1132a32b2419-kube-api-access-dl54h\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.844785    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-lib-modules\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.844889    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.844985    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:32:57.344954715 +0000 UTC m=+2.123437670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.845033    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-iptables-alerter-script\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.845520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.845321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdb15470-3a00-4dc9-9b56-ba56e4661ab1-konnectivity-ca\") pod \"konnectivity-agent-mhjrt\" (UID: \"fdb15470-3a00-4dc9-9b56-ba56e4661ab1\") " pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.847042 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.847015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-tmp\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.847162 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.847071    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-etc-tuned\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.847244 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.847227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdb15470-3a00-4dc9-9b56-ba56e4661ab1-agent-certs\") pod \"konnectivity-agent-mhjrt\" (UID: \"fdb15470-3a00-4dc9-9b56-ba56e4661ab1\") " pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:56.851009 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.850987    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:32:56.851009 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.851008    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:32:56.851209 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.851020    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:32:56.851209 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:56.851079    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:32:57.351065156 +0000 UTC m=+2.129548096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:32:56.853470 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.853444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pztbt\" (UniqueName: \"kubernetes.io/projected/b8649966-7fcf-46ad-af87-392af43bb3df-kube-api-access-pztbt\") pod \"node-resolver-4mb5k\" (UID: \"b8649966-7fcf-46ad-af87-392af43bb3df\") " pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:56.853593 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.853511    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v95vd\" (UniqueName: \"kubernetes.io/projected/dd1ee2a1-6a28-4941-84ce-245b11bdd8e2-kube-api-access-v95vd\") pod \"tuned-ml2m5\" (UID: \"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2\") " pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:56.854418 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.854399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl54h\" (UniqueName: \"kubernetes.io/projected/6a08ba58-2146-46da-bb66-1132a32b2419-kube-api-access-dl54h\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:56.854571 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.854555    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q7kh\" (UniqueName: \"kubernetes.io/projected/2a54ae6b-6aa6-4935-b1cf-7dc71a009793-kube-api-access-7q7kh\") pod \"iptables-alerter-f242z\" (UID: \"2a54ae6b-6aa6-4935-b1cf-7dc71a009793\") " pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:56.962288 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.962171    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:32:56.974163 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:56.974126    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aa96119_a12d_460d_9b53_d2c15e27bc59.slice/crio-c42c33e2bed656e7848bfcfabcdbe8cb651a119552305702e14b3fd21422ae07 WatchSource:0}: Error finding container c42c33e2bed656e7848bfcfabcdbe8cb651a119552305702e14b3fd21422ae07: Status 404 returned error can't find the container with id c42c33e2bed656e7848bfcfabcdbe8cb651a119552305702e14b3fd21422ae07
Jul 17 05:32:56.982981 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.982959    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn"
Jul 17 05:32:56.989317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:56.989288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-47vtj"
Jul 17 05:32:56.995638 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:56.995596    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9dba5b8_53a2_4d53_a7b1_d7d54628d546.slice/crio-40648284c7d13559c4a563129aefcc4c25fb63fccbc4b112dbf9cfe20ca1d2f6 WatchSource:0}: Error finding container 40648284c7d13559c4a563129aefcc4c25fb63fccbc4b112dbf9cfe20ca1d2f6: Status 404 returned error can't find the container with id 40648284c7d13559c4a563129aefcc4c25fb63fccbc4b112dbf9cfe20ca1d2f6
Jul 17 05:32:57.012100 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.012075    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c2f5t"
Jul 17 05:32:57.022247 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:57.022212    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c70031c_daa9_4b4e_a4a4_0c52ac2d0354.slice/crio-8a4292ba6626c8c112da9421f3483d66df4679318c89ec4c6ed547d9584c2f2a WatchSource:0}: Error finding container 8a4292ba6626c8c112da9421f3483d66df4679318c89ec4c6ed547d9584c2f2a: Status 404 returned error can't find the container with id 8a4292ba6626c8c112da9421f3483d66df4679318c89ec4c6ed547d9584c2f2a
Jul 17 05:32:57.027954 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.027928    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fb5fx"
Jul 17 05:32:57.036782 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:57.036752    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b84b7a_c01d_4f80_8605_b8624b272347.slice/crio-e21d16ca92c18ac0af109830e92a52f713374e7d9ad59aca65093b90f152a001 WatchSource:0}: Error finding container e21d16ca92c18ac0af109830e92a52f713374e7d9ad59aca65093b90f152a001: Status 404 returned error can't find the container with id e21d16ca92c18ac0af109830e92a52f713374e7d9ad59aca65093b90f152a001
Jul 17 05:32:57.045425 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.045394    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:32:57.055324 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:57.055294    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb15470_3a00_4dc9_9b56_ba56e4661ab1.slice/crio-6be2700a25629c2c7e7cfbadce611299eca4bd68b285732354caf47cfa031bc4 WatchSource:0}: Error finding container 6be2700a25629c2c7e7cfbadce611299eca4bd68b285732354caf47cfa031bc4: Status 404 returned error can't find the container with id 6be2700a25629c2c7e7cfbadce611299eca4bd68b285732354caf47cfa031bc4
Jul 17 05:32:57.059762 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.059742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ml2m5"
Jul 17 05:32:57.068676 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:57.068648    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1ee2a1_6a28_4941_84ce_245b11bdd8e2.slice/crio-05864b099e02990b4b591bf7dffef22032a665fe74d205d47b5760d3a42440e1 WatchSource:0}: Error finding container 05864b099e02990b4b591bf7dffef22032a665fe74d205d47b5760d3a42440e1: Status 404 returned error can't find the container with id 05864b099e02990b4b591bf7dffef22032a665fe74d205d47b5760d3a42440e1
Jul 17 05:32:57.080516 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.080489    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4mb5k"
Jul 17 05:32:57.086822 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.086798    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f242z"
Jul 17 05:32:57.090993 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:57.090964    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8649966_7fcf_46ad_af87_392af43bb3df.slice/crio-d4cfe5079612168d629ff8bd09baccb0595c710ae82ba449f66275465bde9b54 WatchSource:0}: Error finding container d4cfe5079612168d629ff8bd09baccb0595c710ae82ba449f66275465bde9b54: Status 404 returned error can't find the container with id d4cfe5079612168d629ff8bd09baccb0595c710ae82ba449f66275465bde9b54
Jul 17 05:32:57.096129 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:32:57.096103    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a54ae6b_6aa6_4935_b1cf_7dc71a009793.slice/crio-ab7a7b41913e1600423ca39ff884550e05ac5c1c0f3ef6e680f9e1431c016776 WatchSource:0}: Error finding container ab7a7b41913e1600423ca39ff884550e05ac5c1c0f3ef6e680f9e1431c016776: Status 404 returned error can't find the container with id ab7a7b41913e1600423ca39ff884550e05ac5c1c0f3ef6e680f9e1431c016776
Jul 17 05:32:57.247107 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.247008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:57.247284 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.247172    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:32:57.247284 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.247254    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:32:58.247230679 +0000 UTC m=+3.025713641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:32:57.347927 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.347864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:57.348128 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.348048    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:32:57.348201 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.348135    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:32:58.348113443 +0000 UTC m=+3.126596401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:32:57.448765 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.448720    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:32:57.448999 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.448915    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:32:57.448999 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.448937    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:32:57.448999 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.448951    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:32:57.449279 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:57.449016    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:32:58.448995918 +0000 UTC m=+3.227478861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:32:57.725612 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.725409    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 05:27:56 +0000 UTC" deadline="2027-12-13 18:27:44.571022105 +0000 UTC"
Jul 17 05:32:57.725612 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.725537    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12348h54m46.845489387s"
Jul 17 05:32:57.762980 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.762909    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f242z" event={"ID":"2a54ae6b-6aa6-4935-b1cf-7dc71a009793","Type":"ContainerStarted","Data":"ab7a7b41913e1600423ca39ff884550e05ac5c1c0f3ef6e680f9e1431c016776"}
Jul 17 05:32:57.770377 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.770334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4mb5k" event={"ID":"b8649966-7fcf-46ad-af87-392af43bb3df","Type":"ContainerStarted","Data":"d4cfe5079612168d629ff8bd09baccb0595c710ae82ba449f66275465bde9b54"}
Jul 17 05:32:57.776676 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.776633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ml2m5" event={"ID":"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2","Type":"ContainerStarted","Data":"05864b099e02990b4b591bf7dffef22032a665fe74d205d47b5760d3a42440e1"}
Jul 17 05:32:57.796190 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.796136    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-47vtj" event={"ID":"0833fd9c-124f-4b64-9618-cac8678daad0","Type":"ContainerStarted","Data":"d148c3c1e267fc0447eb9a53c25a1f159c53c229e05de4c4e80e63709b846f9d"}
Jul 17 05:32:57.812033 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.811981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn" event={"ID":"d9dba5b8-53a2-4d53-a7b1-d7d54628d546","Type":"ContainerStarted","Data":"40648284c7d13559c4a563129aefcc4c25fb63fccbc4b112dbf9cfe20ca1d2f6"}
Jul 17 05:32:57.822082 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.822038    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal" event={"ID":"306dc5b5dbae4709c425668aaebe9719","Type":"ContainerStarted","Data":"a09e791f7b0ee305eacdc6bf366e219607c02c44540453669db5c9aa80b7c4f3"}
Jul 17 05:32:57.825672 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.825629    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mhjrt" event={"ID":"fdb15470-3a00-4dc9-9b56-ba56e4661ab1","Type":"ContainerStarted","Data":"6be2700a25629c2c7e7cfbadce611299eca4bd68b285732354caf47cfa031bc4"}
Jul 17 05:32:57.837416 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.837357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fb5fx" event={"ID":"c7b84b7a-c01d-4f80-8605-b8624b272347","Type":"ContainerStarted","Data":"e21d16ca92c18ac0af109830e92a52f713374e7d9ad59aca65093b90f152a001"}
Jul 17 05:32:57.857235 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.857195    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerStarted","Data":"8a4292ba6626c8c112da9421f3483d66df4679318c89ec4c6ed547d9584c2f2a"}
Jul 17 05:32:57.861640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.861597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"c42c33e2bed656e7848bfcfabcdbe8cb651a119552305702e14b3fd21422ae07"}
Jul 17 05:32:57.876615 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:57.876399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal" event={"ID":"75da3d20d9ebca413dcb11475cc9a3b5","Type":"ContainerStarted","Data":"1113dd6d00899a09778d2e0cb7a873aabf0cf912a23f6ed4638462e613574165"}
Jul 17 05:32:58.124294 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.124194    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:32:58.256671 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.256628    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:58.256839 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.256801    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:32:58.256891 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.256873    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:00.256849527 +0000 UTC m=+5.035332487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:32:58.357809 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.357132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:58.357809 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.357306    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:32:58.357809 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.357375    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:00.357355964 +0000 UTC m=+5.135838909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:32:58.459180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.458363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:32:58.459180 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.458572    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:32:58.459180 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.458604    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:32:58.459180 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.458622    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:32:58.459180 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.458692    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:33:00.458669585 +0000 UTC m=+5.237152531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:32:58.726756 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.726575    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 05:27:56 +0000 UTC" deadline="2028-01-07 14:29:14.168819194 +0000 UTC"
Jul 17 05:32:58.726756 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.726631    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12944h56m15.442193857s"
Jul 17 05:32:58.746176 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.746146    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:32:58.746407 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.746276    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:32:58.746512 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.746487    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:32:58.746646 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.746620    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:32:58.746739 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:32:58.746719    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:32:58.746982 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:32:58.746956    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:00.271270 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:00.271223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:00.271731 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.271447    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:00.271731 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.271516    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:04.271497475 +0000 UTC m=+9.049980416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:00.373400 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:00.372725    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:00.373400 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.372890    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:00.373400 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.372951    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:04.372933922 +0000 UTC m=+9.151416863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:00.473449 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:00.473400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:00.473642 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.473629    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:33:00.473702 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.473651    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:33:00.473702 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.473664    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:00.473803 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.473733    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:33:04.473714076 +0000 UTC m=+9.252197040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:00.747348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:00.746424    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:00.747348 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.746567    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:00.747348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:00.747010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:00.747348 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.747120    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:00.747348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:00.747181    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:00.747348 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:00.747254    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:02.746632 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:02.746215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:02.746632 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:02.746248    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:02.746632 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:02.746331    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:02.746632 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:02.746216    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:02.746632 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:02.746424    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:02.746632 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:02.746516    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:04.303771 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:04.303695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:04.304277 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.303915    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:04.304277 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.304003    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:12.303974077 +0000 UTC m=+17.082457018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:04.404505 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:04.404309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:04.404688 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.404515    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:04.404688 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.404579    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:12.404560706 +0000 UTC m=+17.183043644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:04.504848 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:04.504782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:04.505048 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.504957    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:33:04.505048 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.504984    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:33:04.505048 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.505007    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:04.505213 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.505075    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:33:12.505055393 +0000 UTC m=+17.283538349 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:04.746012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:04.745644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:04.746012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:04.745797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:04.746012 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.745795    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:04.746012 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.745936    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:04.746012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:04.745963    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:04.746384 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:04.746043    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:06.746074 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:06.745977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:06.746074 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:06.746027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:06.746715 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:06.746115    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:06.746715 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:06.746198    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:06.746715 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:06.746306    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:06.746715 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:06.746400    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:08.746211 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:08.745995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:08.746629 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:08.745995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:08.746629 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:08.746303    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:08.746629 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:08.746012    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:08.746629 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:08.746376    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:08.746629 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:08.746444    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:10.746235 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:10.746197    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:10.746669 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:10.746197    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:10.746669 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:10.746197    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:10.746669 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:10.746425    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:10.746669 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:10.746320    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:10.746669 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:10.746516    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:12.363647 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:12.363602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:12.364187 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.363749    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:12.364187 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.363822    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:28.363802327 +0000 UTC m=+33.142285284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:12.464790 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:12.464752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:12.464985 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.464948    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:12.465035 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.465027    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:28.465002903 +0000 UTC m=+33.243485863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:12.565255 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:12.565209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:12.565448 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.565398    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:33:12.565448 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.565428    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:33:12.565448 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.565441    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:12.565609 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.565506    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:33:28.56548776 +0000 UTC m=+33.343970698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:12.746475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:12.746386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:12.746475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:12.746415    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:12.746475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:12.746416    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:12.746729 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.746514    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:12.746729 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.746583    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:12.746729 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:12.746631    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:14.746085 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:14.746050    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:14.746649 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:14.746057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:14.746649 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:14.746177    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:14.746649 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:14.746194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:14.746649 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:14.746263    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:14.746649 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:14.746320    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:16.746445 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:16.746407    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:16.746884 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:16.746418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:16.746884 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:16.746557    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:16.746884 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:16.746425    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:16.746884 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:16.746629    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:16.746884 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:16.746721    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:17.935628 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.935410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ml2m5" event={"ID":"dd1ee2a1-6a28-4941-84ce-245b11bdd8e2","Type":"ContainerStarted","Data":"e62fc802ba3bb83bf76d3e55b8f85c8943b64954c42cb439cb188af7949e9856"}
Jul 17 05:33:17.944395 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.944363    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal" event={"ID":"306dc5b5dbae4709c425668aaebe9719","Type":"ContainerStarted","Data":"02a666ddbca8737c529b62ab0a4326f022e32ea0b7475320165090362e7166cf"}
Jul 17 05:33:17.945876 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.945832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fb5fx" event={"ID":"c7b84b7a-c01d-4f80-8605-b8624b272347","Type":"ContainerStarted","Data":"c2be47272c97cde49b93270b6b5c0ca680c05ad4d611ab1fbd194f1e04ab36dc"}
Jul 17 05:33:17.947532 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.947510    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"526c9d30ad9cfd71120a75f61442bc4c6e0f2c76b9accde20817a67382b50a9b"}
Jul 17 05:33:17.947646 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.947573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"70bf1850bb2943997b3785360332501cd7d70d7a0bdd9ff9b9626a9675b811ff"}
Jul 17 05:33:17.972506 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.968670    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fb5fx" podStartSLOduration=1.593802562 podStartE2EDuration="21.968649453s" totalImagesPullingTime="20.374846891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:17.968168384 +0000 UTC m=+22.746651344" watchObservedRunningTime="2026-07-17 05:33:17.968649453 +0000 UTC m=+22.747132417"
Jul 17 05:33:17.972506 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.969016    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ml2m5" podStartSLOduration=1.637804934 podStartE2EDuration="21.96900671s" totalImagesPullingTime="20.331201776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:17.951324576 +0000 UTC m=+22.729807535" watchObservedRunningTime="2026-07-17 05:33:17.96900671 +0000 UTC m=+22.747489672"
Jul 17 05:33:17.984191 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:17.984141    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-189.ec2.internal" podStartSLOduration=21.984125501 podStartE2EDuration="21.984125501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:17.983919376 +0000 UTC m=+22.762402336" watchObservedRunningTime="2026-07-17 05:33:17.984125501 +0000 UTC m=+22.762608461"
Jul 17 05:33:18.746265 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.746056    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:18.746548 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.746070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:18.746548 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.746102    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:18.746548 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:18.746393    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:18.746548 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:18.746430    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:18.746548 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:18.746500    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:18.950998 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.950954    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mhjrt" event={"ID":"fdb15470-3a00-4dc9-9b56-ba56e4661ab1","Type":"ContainerStarted","Data":"b4ecbbe502218bb2e42d23190421b3691601fa304fbf30ceb55a5081606e65db"}
Jul 17 05:33:18.952528 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.952498    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c70031c-daa9-4b4e-a4a4-0c52ac2d0354" containerID="3818ae9cec1c07e88cea575e3f84daf2eeb0e3ecb9426f6bc2c7f29a6ad60e2f" exitCode=0
Jul 17 05:33:18.952659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.952588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerDied","Data":"3818ae9cec1c07e88cea575e3f84daf2eeb0e3ecb9426f6bc2c7f29a6ad60e2f"}
Jul 17 05:33:18.955709 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.955680    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"657c814582cc0efbd9518d7d94eea9334f7fd7eb981c61f4fba7520af7c861d5"}
Jul 17 05:33:18.955812 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.955766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"cf6cf51ec96906ca99ba5a989d723bab70336e68a2f15ae724c8bb73fe35ee2a"}
Jul 17 05:33:18.955898 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.955816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"4c230374377c2d2e9c9e49f088087db76dfe57dec08e592b03586d92b906bd29"}
Jul 17 05:33:18.955898 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.955845    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"81b048a7daaaaf3a074f2f81bcdd26bef24ed7ba75fb94b539c91a6b76360814"}
Jul 17 05:33:18.957148 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.957126    2559 generic.go:356] "Generic (PLEG): container finished" podID="75da3d20d9ebca413dcb11475cc9a3b5" containerID="9e1f2e33ba14cf1818fb5aaa24e46dd06b8fcb23ebbe0bcdeddbe25c8f02f25a" exitCode=0
Jul 17 05:33:18.957258 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.957189    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal" event={"ID":"75da3d20d9ebca413dcb11475cc9a3b5","Type":"ContainerDied","Data":"9e1f2e33ba14cf1818fb5aaa24e46dd06b8fcb23ebbe0bcdeddbe25c8f02f25a"}
Jul 17 05:33:18.959154 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.958723    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f242z" event={"ID":"2a54ae6b-6aa6-4935-b1cf-7dc71a009793","Type":"ContainerStarted","Data":"6142b61980a7e3ea3f16dc736a5bc843001c23a22928a5067a5d1eb57f743314"}
Jul 17 05:33:18.962195 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.962157    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4mb5k" event={"ID":"b8649966-7fcf-46ad-af87-392af43bb3df","Type":"ContainerStarted","Data":"f49c2fc57067b5b28e76869dd8e50dfdb52a8c70a29554dcf1077633de7823f9"}
Jul 17 05:33:18.963602 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.963574    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-47vtj" event={"ID":"0833fd9c-124f-4b64-9618-cac8678daad0","Type":"ContainerStarted","Data":"57e68834da5b52c56caae46430ecb2de585ac38adb09c1ee693791b4ac4c4463"}
Jul 17 05:33:18.965005 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.964980    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn" event={"ID":"d9dba5b8-53a2-4d53-a7b1-d7d54628d546","Type":"ContainerStarted","Data":"bb9ef7b48ff37f2c46b2614fb1b0cbfda68b97b1b7830ff3260a7822eea4af51"}
Jul 17 05:33:18.965558 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.965522    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mhjrt" podStartSLOduration=2.623158641 podStartE2EDuration="22.965507573s" totalImagesPullingTime="20.342348932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:18.964790768 +0000 UTC m=+23.743273730" watchObservedRunningTime="2026-07-17 05:33:18.965507573 +0000 UTC m=+23.743990532"
Jul 17 05:33:18.992902 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:18.992818    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-47vtj" podStartSLOduration=2.596807743 podStartE2EDuration="22.992797331s" totalImagesPullingTime="20.395989588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:18.992789638 +0000 UTC m=+23.771272601" watchObservedRunningTime="2026-07-17 05:33:18.992797331 +0000 UTC m=+23.771280292"
Jul 17 05:33:19.030804 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.030756    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4mb5k" podStartSLOduration=2.723862767 podStartE2EDuration="23.030737529s" totalImagesPullingTime="20.306874762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:19.030503002 +0000 UTC m=+23.808985962" watchObservedRunningTime="2026-07-17 05:33:19.030737529 +0000 UTC m=+23.809220490"
Jul 17 05:33:19.048196 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.048133    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f242z" podStartSLOduration=2.746469695 podStartE2EDuration="23.048115797s" totalImagesPullingTime="20.301646102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:19.047993319 +0000 UTC m=+23.826476279" watchObservedRunningTime="2026-07-17 05:33:19.048115797 +0000 UTC m=+23.826598802"
Jul 17 05:33:19.328776 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.328748    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 05:33:19.707378 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.707195    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T05:33:19.32876922Z","UUID":"65a0dccf-3423-44bd-8564-b74374d7529d","Handler":null,"Name":"","Endpoint":""}
Jul 17 05:33:19.709244 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.709215    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 05:33:19.709388 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.709267    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 05:33:19.969592 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.969498    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal" event={"ID":"75da3d20d9ebca413dcb11475cc9a3b5","Type":"ContainerStarted","Data":"8bfbdafca705b33ca68fd4936864205499990b1c12d92a8220ec00ddbaff63c3"}
Jul 17 05:33:19.972152 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.972066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn" event={"ID":"d9dba5b8-53a2-4d53-a7b1-d7d54628d546","Type":"ContainerStarted","Data":"8f52094dcb40bb4278574539c3a3976bc87bdd1d62f3b7c2dd3853961ebfaf4c"}
Jul 17 05:33:19.987308 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:19.987251    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-189.ec2.internal" podStartSLOduration=23.987229199 podStartE2EDuration="23.987229199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:19.986229964 +0000 UTC m=+24.764712926" watchObservedRunningTime="2026-07-17 05:33:19.987229199 +0000 UTC m=+24.765712164"
Jul 17 05:33:20.745696 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:20.745606    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:20.745696 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:20.745635    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:20.745696 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:20.745606    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:20.746032 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:20.745755    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:20.746032 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:20.745930    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:20.746134 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:20.746070    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:20.977555 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:20.977514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn" event={"ID":"d9dba5b8-53a2-4d53-a7b1-d7d54628d546","Type":"ContainerStarted","Data":"d6e58d7aa3ac4be1dddcf41a0983523b270995141c06d3fa940f9f3e68e8d42a"}
Jul 17 05:33:20.981918 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:20.981880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"49b0831b3b1055195f93e3dd8f19ba1a46d931ba903e85c5083553ec05731a31"}
Jul 17 05:33:20.995683 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:20.995600    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m7zn" podStartSLOduration=2.329786492 podStartE2EDuration="24.995574398s" totalImagesPullingTime="22.665787906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:20.994900139 +0000 UTC m=+25.773383099" watchObservedRunningTime="2026-07-17 05:33:20.995574398 +0000 UTC m=+25.774057360"
Jul 17 05:33:21.243483 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:21.243388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:33:21.244198 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:21.244155    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:33:21.985411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:21.985370    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:33:21.985967 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:21.985946    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mhjrt"
Jul 17 05:33:22.745958 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:22.745919    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:22.745958 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:22.745951    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:22.746215 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:22.745951    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:22.746215 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:22.746064    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:22.746215 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:22.746127    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:22.746215 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:22.746199    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:23.990472 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:23.990441    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c70031c-daa9-4b4e-a4a4-0c52ac2d0354" containerID="094f1477904ccaae1dd8ef80f97f0eb2bb3e9b4d84e1dd85acd332f1a6d0beef" exitCode=0
Jul 17 05:33:23.991395 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:23.990513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerDied","Data":"094f1477904ccaae1dd8ef80f97f0eb2bb3e9b4d84e1dd85acd332f1a6d0beef"}
Jul 17 05:33:23.994346 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:23.994321    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" event={"ID":"2aa96119-a12d-460d-9b53-d2c15e27bc59","Type":"ContainerStarted","Data":"b54704db787677285ac33ab0a21aab40b7c95f0760a7375f1af33076fafc66ea"}
Jul 17 05:33:24.031605 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.031544    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz" podStartSLOduration=7.38462288 podStartE2EDuration="28.031526313s" totalImagesPullingTime="20.646903433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:24.030535923 +0000 UTC m=+28.809018895" watchObservedRunningTime="2026-07-17 05:33:24.031526313 +0000 UTC m=+28.810009273"
Jul 17 05:33:24.745903 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.745679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:24.746068 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.745691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:24.746068 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:24.745998    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:24.746145 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:24.746089    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:24.746145 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.745757    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:24.746236 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:24.746217    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:24.957502 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.957367    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7gz6h"]
Jul 17 05:33:24.961228 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.960534    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rk2tl"]
Jul 17 05:33:24.961538 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.961363    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mt2jm"]
Jul 17 05:33:24.996701 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.996599    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:24.996701 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.996624    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:24.996701 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:24.996595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:24.997177 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:24.996725    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:25.005416 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:25.005099    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:25.005416 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:25.005213    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:25.005416 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:25.005252    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:33:25.005416 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:25.005319    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:33:25.005416 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:25.005363    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:33:25.014379 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:25.014351    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:33:25.014538 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:25.014455    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:33:26.000457 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:26.000419    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c70031c-daa9-4b4e-a4a4-0c52ac2d0354" containerID="e0a5fac5bf96350640c57c62143ed9ac05edf0ebc8e18e89e92d80d742608103" exitCode=0
Jul 17 05:33:26.000979 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:26.000499    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerDied","Data":"e0a5fac5bf96350640c57c62143ed9ac05edf0ebc8e18e89e92d80d742608103"}
Jul 17 05:33:26.745938 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:26.745901    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:26.745938 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:26.745919    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:26.746225 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:26.745903    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:26.746225 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:26.746027    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:26.746225 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:26.746124    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:26.746225 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:26.746220    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:28.010687 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.010641    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c70031c-daa9-4b4e-a4a4-0c52ac2d0354" containerID="7106dcf3ba0c1a9ccb4203c73786d713760e5c352d77a9aacb5ae63da8677fd9" exitCode=0
Jul 17 05:33:28.011128 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.010718    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerDied","Data":"7106dcf3ba0c1a9ccb4203c73786d713760e5c352d77a9aacb5ae63da8677fd9"}
Jul 17 05:33:28.376358 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.376056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:28.376358 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.376209    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:28.376566 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.376377    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret podName:93a50d18-6fa3-4037-8174-4ff5a09f6fb8 nodeName:}" failed. No retries permitted until 2026-07-17 05:34:00.376357591 +0000 UTC m=+65.154840545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret") pod "global-pull-secret-syncer-7gz6h" (UID: "93a50d18-6fa3-4037-8174-4ff5a09f6fb8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:33:28.477257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.477218    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:28.477475 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.477391    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:28.477542 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.477483    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs podName:6a08ba58-2146-46da-bb66-1132a32b2419 nodeName:}" failed. No retries permitted until 2026-07-17 05:34:00.477461935 +0000 UTC m=+65.255944875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs") pod "network-metrics-daemon-rk2tl" (UID: "6a08ba58-2146-46da-bb66-1132a32b2419") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:33:28.578607 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.578563    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:28.578802 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.578727    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:33:28.578802 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.578743    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:33:28.578802 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.578752    2559 projected.go:196] Error preparing data for projected volume kube-api-access-cs6js for pod openshift-network-diagnostics/network-check-target-mt2jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:28.579007 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.578810    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js podName:0ac4b401-3f2b-4b26-ac52-0d87aa4f845c nodeName:}" failed. No retries permitted until 2026-07-17 05:34:00.578791927 +0000 UTC m=+65.357274880 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6js" (UniqueName: "kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js") pod "network-check-target-mt2jm" (UID: "0ac4b401-3f2b-4b26-ac52-0d87aa4f845c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:33:28.746288 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.746249    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:28.746488 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.746262    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:28.746488 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.746386    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:28.746488 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:28.746273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:28.746488 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.746443    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:28.746690 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:28.746501    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:30.746734 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:30.746697    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:30.747205 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:30.746693    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:30.747205 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:30.746838    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rk2tl" podUID="6a08ba58-2146-46da-bb66-1132a32b2419"
Jul 17 05:33:30.747205 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:30.746706    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:30.747205 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:30.746958    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mt2jm" podUID="0ac4b401-3f2b-4b26-ac52-0d87aa4f845c"
Jul 17 05:33:30.747205 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:30.747057    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gz6h" podUID="93a50d18-6fa3-4037-8174-4ff5a09f6fb8"
Jul 17 05:33:31.431559 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.431476    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-189.ec2.internal" event="NodeReady"
Jul 17 05:33:31.431713 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.431639    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 05:33:31.465971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.465934    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69787f6b6f-8vhgd"]
Jul 17 05:33:31.468298 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.468265    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.470738 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.470696    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 05:33:31.470738 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.470723    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mxhcm\""
Jul 17 05:33:31.470983 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.470726    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 05:33:31.470983 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.470926    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 05:33:31.476159 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.476130    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 05:33:31.481679 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.481652    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69787f6b6f-8vhgd"]
Jul 17 05:33:31.484441 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.484410    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9m5m5"]
Jul 17 05:33:31.486408 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.486369    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hzrw2"]
Jul 17 05:33:31.486983 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.486639    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.488223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.488201    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.489623 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.489477    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 05:33:31.489623 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.489480    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qq4z7\""
Jul 17 05:33:31.489623 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.489567    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 05:33:31.490340 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.490315    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 05:33:31.490583 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.490561    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 05:33:31.490686 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.490629    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 05:33:31.490964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.490945    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7j7x7\""
Jul 17 05:33:31.495277 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.495251    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzrw2"]
Jul 17 05:33:31.498201 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.498138    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9m5m5"]
Jul 17 05:33:31.602093 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53c331fb-d346-4c18-b9eb-d0b8a8825437-config-volume\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.602290 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-installation-pull-secrets\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602290 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c331fb-d346-4c18-b9eb-d0b8a8825437-tmp-dir\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.602290 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2qwq\" (UniqueName: \"kubernetes.io/projected/53c331fb-d346-4c18-b9eb-d0b8a8825437-kube-api-access-w2qwq\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.602411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602304    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-image-registry-private-configuration\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602336    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.602411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602364    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-bound-sa-token\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae-cert\") pod \"ingress-canary-hzrw2\" (UID: \"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae\") " pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.602572 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602419    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602572 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-trusted-ca\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602572 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602526    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k77ct\" (UniqueName: \"kubernetes.io/projected/d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae-kube-api-access-k77ct\") pod \"ingress-canary-hzrw2\" (UID: \"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae\") " pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.602710 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602591    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20673ee8-572c-4ad1-8a61-27f45fdf36bf-ca-trust-extracted\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602710 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-certificates\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.602710 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.602645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8kg7\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-kube-api-access-j8kg7\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.703814 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703714    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k77ct\" (UniqueName: \"kubernetes.io/projected/d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae-kube-api-access-k77ct\") pod \"ingress-canary-hzrw2\" (UID: \"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae\") " pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.703814 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703780    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20673ee8-572c-4ad1-8a61-27f45fdf36bf-ca-trust-extracted\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.703814 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-certificates\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8kg7\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-kube-api-access-j8kg7\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53c331fb-d346-4c18-b9eb-d0b8a8825437-config-volume\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-installation-pull-secrets\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.703969    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c331fb-d346-4c18-b9eb-d0b8a8825437-tmp-dir\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2qwq\" (UniqueName: \"kubernetes.io/projected/53c331fb-d346-4c18-b9eb-d0b8a8825437-kube-api-access-w2qwq\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-image-registry-private-configuration\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704073    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-bound-sa-token\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae-cert\") pod \"ingress-canary-hzrw2\" (UID: \"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae\") " pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.704534 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704534 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-trusted-ca\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.704534 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.704308    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20673ee8-572c-4ad1-8a61-27f45fdf36bf-ca-trust-extracted\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.705059 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:31.705027    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:33:31.705184 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.705074    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-certificates\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.705264 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:31.705105    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls podName:53c331fb-d346-4c18-b9eb-d0b8a8825437 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:32.205086979 +0000 UTC m=+36.983569934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls") pod "dns-default-9m5m5" (UID: "53c331fb-d346-4c18-b9eb-d0b8a8825437") : secret "dns-default-metrics-tls" not found
Jul 17 05:33:31.706056 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.706028    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-trusted-ca\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.707561 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.706366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53c331fb-d346-4c18-b9eb-d0b8a8825437-config-volume\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.707561 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.707179    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c331fb-d346-4c18-b9eb-d0b8a8825437-tmp-dir\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.707561 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:31.707505    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:33:31.707561 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:31.707565    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69787f6b6f-8vhgd: secret "image-registry-tls" not found
Jul 17 05:33:31.710226 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:31.710172    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls podName:20673ee8-572c-4ad1-8a61-27f45fdf36bf nodeName:}" failed. No retries permitted until 2026-07-17 05:33:32.210118037 +0000 UTC m=+36.988600986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls") pod "image-registry-69787f6b6f-8vhgd" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf") : secret "image-registry-tls" not found
Jul 17 05:33:31.711409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.711383    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-installation-pull-secrets\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.711603 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.711576    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-image-registry-private-configuration\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.711911 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.711712    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae-cert\") pod \"ingress-canary-hzrw2\" (UID: \"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae\") " pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.714288 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.714266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k77ct\" (UniqueName: \"kubernetes.io/projected/d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae-kube-api-access-k77ct\") pod \"ingress-canary-hzrw2\" (UID: \"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae\") " pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:31.714288 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.714277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-bound-sa-token\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.714435 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.714387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8kg7\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-kube-api-access-j8kg7\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:31.715498 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.715479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2qwq\" (UniqueName: \"kubernetes.io/projected/53c331fb-d346-4c18-b9eb-d0b8a8825437-kube-api-access-w2qwq\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:31.806998 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:31.806952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzrw2"
Jul 17 05:33:32.146301 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.146033    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzrw2"]
Jul 17 05:33:32.150219 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:33:32.150189    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b17b3a_cc6c_47fe_b73e_c7a6966e34ae.slice/crio-1c9df9f62bcdae5aa8028fe9b286288637eef717ad60a3a35869a0176cc38b03 WatchSource:0}: Error finding container 1c9df9f62bcdae5aa8028fe9b286288637eef717ad60a3a35869a0176cc38b03: Status 404 returned error can't find the container with id 1c9df9f62bcdae5aa8028fe9b286288637eef717ad60a3a35869a0176cc38b03
Jul 17 05:33:32.209072 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.209024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:32.209276 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:32.209164    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:33:32.209276 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:32.209245    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls podName:53c331fb-d346-4c18-b9eb-d0b8a8825437 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:33.209225409 +0000 UTC m=+37.987708349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls") pod "dns-default-9m5m5" (UID: "53c331fb-d346-4c18-b9eb-d0b8a8825437") : secret "dns-default-metrics-tls" not found
Jul 17 05:33:32.310037 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.309993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:32.310209 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:32.310166    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:33:32.310209 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:32.310186    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69787f6b6f-8vhgd: secret "image-registry-tls" not found
Jul 17 05:33:32.310285 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:32.310247    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls podName:20673ee8-572c-4ad1-8a61-27f45fdf36bf nodeName:}" failed. No retries permitted until 2026-07-17 05:33:33.310229988 +0000 UTC m=+38.088712942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls") pod "image-registry-69787f6b6f-8vhgd" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf") : secret "image-registry-tls" not found
Jul 17 05:33:32.746307 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.746268    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:33:32.746490 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.746312    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:33:32.746490 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.746268    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:33:32.749833 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.749811    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 05:33:32.749833 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.749828    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 05:33:32.750070 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.749893    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44td2\""
Jul 17 05:33:32.750070 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.749944    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x8c5l\""
Jul 17 05:33:32.750070 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.749963    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 05:33:32.750070 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:32.750016    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 05:33:33.025165 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:33.025085    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzrw2" event={"ID":"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae","Type":"ContainerStarted","Data":"1c9df9f62bcdae5aa8028fe9b286288637eef717ad60a3a35869a0176cc38b03"}
Jul 17 05:33:33.028429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:33.028395    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c70031c-daa9-4b4e-a4a4-0c52ac2d0354" containerID="62535af9168598a41343c32abe258a5250bfacfd99d6a3594d0e37f8ecc3fadf" exitCode=0
Jul 17 05:33:33.028605 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:33.028476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerDied","Data":"62535af9168598a41343c32abe258a5250bfacfd99d6a3594d0e37f8ecc3fadf"}
Jul 17 05:33:33.217020 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:33.216982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:33.217217 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:33.217153    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:33:33.217284 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:33.217227    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls podName:53c331fb-d346-4c18-b9eb-d0b8a8825437 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:35.217210696 +0000 UTC m=+39.995693633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls") pod "dns-default-9m5m5" (UID: "53c331fb-d346-4c18-b9eb-d0b8a8825437") : secret "dns-default-metrics-tls" not found
Jul 17 05:33:33.318568 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:33.318480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:33.318736 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:33.318641    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:33:33.318736 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:33.318660    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69787f6b6f-8vhgd: secret "image-registry-tls" not found
Jul 17 05:33:33.318736 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:33.318719    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls podName:20673ee8-572c-4ad1-8a61-27f45fdf36bf nodeName:}" failed. No retries permitted until 2026-07-17 05:33:35.318699574 +0000 UTC m=+40.097182513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls") pod "image-registry-69787f6b6f-8vhgd" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf") : secret "image-registry-tls" not found
Jul 17 05:33:34.035300 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.035255    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c70031c-daa9-4b4e-a4a4-0c52ac2d0354" containerID="bf0dc0b7261699d78101362b56c4d762509d0fff45b81c39e948a5ebe5467d51" exitCode=0
Jul 17 05:33:34.035778 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.035340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerDied","Data":"bf0dc0b7261699d78101362b56c4d762509d0fff45b81c39e948a5ebe5467d51"}
Jul 17 05:33:34.675144 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.675112    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:34.760689 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.760410    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:34.826595 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.826501    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:34.913515 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.913472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:34.979502 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:34.979469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:35.041109 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.041071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" event={"ID":"3c70031c-daa9-4b4e-a4a4-0c52ac2d0354","Type":"ContainerStarted","Data":"02e6f1fd2f692ceb698e3997ad7326dbc2bc46d589b5e5bbca1c49f398257e73"}
Jul 17 05:33:35.042486 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.042457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzrw2" event={"ID":"d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae","Type":"ContainerStarted","Data":"3bf9a5be137b52499fd3f91abb6212309f73c1b66364c8097adb5bdd1bdca3aa"}
Jul 17 05:33:35.066838 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.066773    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c2f5t" podStartSLOduration=8.664808594 podStartE2EDuration="39.066748446s" totalImagesPullingTime="30.401939852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:35.064779912 +0000 UTC m=+39.843262873" watchObservedRunningTime="2026-07-17 05:33:35.066748446 +0000 UTC m=+39.845231407"
Jul 17 05:33:35.079626 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.079503    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hzrw2" podStartSLOduration=1.627101474 podStartE2EDuration="4.079483274s" totalImagesPullingTime="2.4523818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:35.079417963 +0000 UTC m=+39.857900924" watchObservedRunningTime="2026-07-17 05:33:35.079483274 +0000 UTC m=+39.857966235"
Jul 17 05:33:35.192425 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.192384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:35.236589 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.236553    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:35.236711 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:35.236670    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:33:35.236744 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:35.236729    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls podName:53c331fb-d346-4c18-b9eb-d0b8a8825437 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:39.23671646 +0000 UTC m=+44.015199399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls") pod "dns-default-9m5m5" (UID: "53c331fb-d346-4c18-b9eb-d0b8a8825437") : secret "dns-default-metrics-tls" not found
Jul 17 05:33:35.292355 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.292307    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:35.337467 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:35.337367    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:35.337622 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:35.337548    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:33:35.337622 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:35.337568    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69787f6b6f-8vhgd: secret "image-registry-tls" not found
Jul 17 05:33:35.337732 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:35.337634    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls podName:20673ee8-572c-4ad1-8a61-27f45fdf36bf nodeName:}" failed. No retries permitted until 2026-07-17 05:33:39.337614737 +0000 UTC m=+44.116097693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls") pod "image-registry-69787f6b6f-8vhgd" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf") : secret "image-registry-tls" not found
Jul 17 05:33:36.727290 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:36.727251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:36.822383 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:36.822338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:36.918637 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:36.918602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:39.267605 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:39.267554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:39.268073 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:39.267718    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:33:39.268073 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:39.267783    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls podName:53c331fb-d346-4c18-b9eb-d0b8a8825437 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:47.267766765 +0000 UTC m=+52.046249702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls") pod "dns-default-9m5m5" (UID: "53c331fb-d346-4c18-b9eb-d0b8a8825437") : secret "dns-default-metrics-tls" not found
Jul 17 05:33:39.365452 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:39.365414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:39.368512 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:39.368488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:39.368644 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:39.368627    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:33:39.368682 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:39.368646    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69787f6b6f-8vhgd: secret "image-registry-tls" not found
Jul 17 05:33:39.368715 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:39.368698    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls podName:20673ee8-572c-4ad1-8a61-27f45fdf36bf nodeName:}" failed. No retries permitted until 2026-07-17 05:33:47.368684481 +0000 UTC m=+52.147167419 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls") pod "image-registry-69787f6b6f-8vhgd" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf") : secret "image-registry-tls" not found
Jul 17 05:33:39.465978 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:39.465945    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:41.655925 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.655888    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"]
Jul 17 05:33:41.687476 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.687445    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"]
Jul 17 05:33:41.687652 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.687559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.690098 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.690071    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 17 05:33:41.691309 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.691282    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 05:33:41.691450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.691297    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kdnlm\""
Jul 17 05:33:41.691450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.691422    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 05:33:41.691450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.691422    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 17 05:33:41.711440 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.711402    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-czqrr"]
Jul 17 05:33:41.711608 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.711531    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:41.714278 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.714251    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 17 05:33:41.714278 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.714271    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p6skv\""
Jul 17 05:33:41.714472 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.714287    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 17 05:33:41.714472 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.714264    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:33:41.731526 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.731482    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"]
Jul 17 05:33:41.731526 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.731531    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-czqrr"]
Jul 17 05:33:41.731739 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.731545    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"]
Jul 17 05:33:41.731739 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.731645    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.742064 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.742034    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 17 05:33:41.742210 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.742118    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 17 05:33:41.742338 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.742324    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 05:33:41.742463 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.742447    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 05:33:41.742586 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.742570    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pcwtg\""
Jul 17 05:33:41.747398 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.747378    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 17 05:33:41.762186 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.762138    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"]
Jul 17 05:33:41.783582 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.783552    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"]
Jul 17 05:33:41.783745 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.783671    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"
Jul 17 05:33:41.786260 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.786240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r5cds\""
Jul 17 05:33:41.786372 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.786292    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pgs8\" (UniqueName: \"kubernetes.io/projected/a1df72a6-3180-44b4-a30d-a3e1917adccb-kube-api-access-9pgs8\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.786433 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.786386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.786484 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.786451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65mpq\" (UniqueName: \"kubernetes.io/projected/a1935ff2-b7b3-4307-bce9-aa549479b4c9-kube-api-access-65mpq\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:41.786553 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.786533    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:41.786661 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.786571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a1df72a6-3180-44b4-a30d-a3e1917adccb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.803698 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.803663    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"]
Jul 17 05:33:41.803698 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.803700    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"]
Jul 17 05:33:41.803962 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.803773    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"
Jul 17 05:33:41.805968 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.805942    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:33:41.806319 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.806301    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 17 05:33:41.806376 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.806345    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nmjr4\""
Jul 17 05:33:41.887584 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887549    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/34afd671-dce9-4fce-8fa0-c265ca199e21-serving-cert\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.887584 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887591    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34afd671-dce9-4fce-8fa0-c265ca199e21-service-ca-bundle\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.887896 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34afd671-dce9-4fce-8fa0-c265ca199e21-tmp\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.887896 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887661    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v25n7\" (UniqueName: \"kubernetes.io/projected/34afd671-dce9-4fce-8fa0-c265ca199e21-kube-api-access-v25n7\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.887896 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887804    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pgs8\" (UniqueName: \"kubernetes.io/projected/a1df72a6-3180-44b4-a30d-a3e1917adccb-kube-api-access-9pgs8\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.887896 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887871    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.888065 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887901    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/34afd671-dce9-4fce-8fa0-c265ca199e21-snapshots\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.888065 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.887944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bhd2\" (UniqueName: \"kubernetes.io/projected/7f766e54-ad49-4a16-a8a2-71e727dbeab2-kube-api-access-9bhd2\") pod \"volume-data-source-validator-78cf4c6f6-7bwdw\" (UID: \"7f766e54-ad49-4a16-a8a2-71e727dbeab2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"
Jul 17 05:33:41.888065 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:41.888009    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:41.888065 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.888021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34afd671-dce9-4fce-8fa0-c265ca199e21-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:41.888074    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls podName:a1df72a6-3180-44b4-a30d-a3e1917adccb nodeName:}" failed. No retries permitted until 2026-07-17 05:33:42.388054855 +0000 UTC m=+47.166537793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-74mbk" (UID: "a1df72a6-3180-44b4-a30d-a3e1917adccb") : secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.888101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rns\" (UniqueName: \"kubernetes.io/projected/755460e4-929f-4f3f-9b39-da00e1e93048-kube-api-access-d6rns\") pod \"network-check-source-7bc5b6bf85-nvnxf\" (UID: \"755460e4-929f-4f3f-9b39-da00e1e93048\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.888171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.888201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a1df72a6-3180-44b4-a30d-a3e1917adccb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:41.888237    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.888263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65mpq\" (UniqueName: \"kubernetes.io/projected/a1935ff2-b7b3-4307-bce9-aa549479b4c9-kube-api-access-65mpq\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:41.888356 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:41.888284    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls podName:a1935ff2-b7b3-4307-bce9-aa549479b4c9 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:42.388265856 +0000 UTC m=+47.166748806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4stv9" (UID: "a1935ff2-b7b3-4307-bce9-aa549479b4c9") : secret "samples-operator-tls" not found
Jul 17 05:33:41.889048 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.889025    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a1df72a6-3180-44b4-a30d-a3e1917adccb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.900795 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.900761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pgs8\" (UniqueName: \"kubernetes.io/projected/a1df72a6-3180-44b4-a30d-a3e1917adccb-kube-api-access-9pgs8\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:41.901017 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.900995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65mpq\" (UniqueName: \"kubernetes.io/projected/a1935ff2-b7b3-4307-bce9-aa549479b4c9-kube-api-access-65mpq\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:41.985550 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.985516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:41.989525 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v25n7\" (UniqueName: \"kubernetes.io/projected/34afd671-dce9-4fce-8fa0-c265ca199e21-kube-api-access-v25n7\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.989655 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/34afd671-dce9-4fce-8fa0-c265ca199e21-snapshots\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.989725 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bhd2\" (UniqueName: \"kubernetes.io/projected/7f766e54-ad49-4a16-a8a2-71e727dbeab2-kube-api-access-9bhd2\") pod \"volume-data-source-validator-78cf4c6f6-7bwdw\" (UID: \"7f766e54-ad49-4a16-a8a2-71e727dbeab2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"
Jul 17 05:33:41.989773 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34afd671-dce9-4fce-8fa0-c265ca199e21-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.989773 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989767    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rns\" (UniqueName: \"kubernetes.io/projected/755460e4-929f-4f3f-9b39-da00e1e93048-kube-api-access-d6rns\") pod \"network-check-source-7bc5b6bf85-nvnxf\" (UID: \"755460e4-929f-4f3f-9b39-da00e1e93048\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"
Jul 17 05:33:41.989916 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/34afd671-dce9-4fce-8fa0-c265ca199e21-serving-cert\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.989988 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34afd671-dce9-4fce-8fa0-c265ca199e21-service-ca-bundle\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.989988 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.989976    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34afd671-dce9-4fce-8fa0-c265ca199e21-tmp\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.990201 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.990181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/34afd671-dce9-4fce-8fa0-c265ca199e21-snapshots\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.990267 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.990253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34afd671-dce9-4fce-8fa0-c265ca199e21-tmp\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.990506 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.990487    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34afd671-dce9-4fce-8fa0-c265ca199e21-service-ca-bundle\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.990599 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.990582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34afd671-dce9-4fce-8fa0-c265ca199e21-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.992262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.992244    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/34afd671-dce9-4fce-8fa0-c265ca199e21-serving-cert\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:41.998674 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:41.998653    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rns\" (UniqueName: \"kubernetes.io/projected/755460e4-929f-4f3f-9b39-da00e1e93048-kube-api-access-d6rns\") pod \"network-check-source-7bc5b6bf85-nvnxf\" (UID: \"755460e4-929f-4f3f-9b39-da00e1e93048\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"
Jul 17 05:33:42.001153 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.001129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v25n7\" (UniqueName: \"kubernetes.io/projected/34afd671-dce9-4fce-8fa0-c265ca199e21-kube-api-access-v25n7\") pod \"insights-operator-555d9d7885-czqrr\" (UID: \"34afd671-dce9-4fce-8fa0-c265ca199e21\") " pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:42.008382 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.008356    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bhd2\" (UniqueName: \"kubernetes.io/projected/7f766e54-ad49-4a16-a8a2-71e727dbeab2-kube-api-access-9bhd2\") pod \"volume-data-source-validator-78cf4c6f6-7bwdw\" (UID: \"7f766e54-ad49-4a16-a8a2-71e727dbeab2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"
Jul 17 05:33:42.040388 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.040347    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-czqrr"
Jul 17 05:33:42.076390 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.076359    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:42.093274 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.093240    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"
Jul 17 05:33:42.112613 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.112579    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"
Jul 17 05:33:42.187614 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.187572    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:42.232150 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.232067    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-czqrr"]
Jul 17 05:33:42.236768 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:33:42.236701    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34afd671_dce9_4fce_8fa0_c265ca199e21.slice/crio-4ece2aa3f7339ca5bb8bfdfc2c55a8766d80afc4f474ebf70de611c37679f8c7 WatchSource:0}: Error finding container 4ece2aa3f7339ca5bb8bfdfc2c55a8766d80afc4f474ebf70de611c37679f8c7: Status 404 returned error can't find the container with id 4ece2aa3f7339ca5bb8bfdfc2c55a8766d80afc4f474ebf70de611c37679f8c7
Jul 17 05:33:42.255173 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.255131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:42.279976 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.279946    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf"]
Jul 17 05:33:42.282648 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:33:42.282619    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755460e4_929f_4f3f_9b39_da00e1e93048.slice/crio-08af66b35c7d12e9ff0b5d019fb0c4d32010a0556b7f2aa43cda859c2d4b59b3 WatchSource:0}: Error finding container 08af66b35c7d12e9ff0b5d019fb0c4d32010a0556b7f2aa43cda859c2d4b59b3: Status 404 returned error can't find the container with id 08af66b35c7d12e9ff0b5d019fb0c4d32010a0556b7f2aa43cda859c2d4b59b3
Jul 17 05:33:42.294598 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.294571    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw"]
Jul 17 05:33:42.298108 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:33:42.298083    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f766e54_ad49_4a16_a8a2_71e727dbeab2.slice/crio-46f852d8a18e29ddf6fddac83d0f4315303d88273e607449d9ee2648e907ff20 WatchSource:0}: Error finding container 46f852d8a18e29ddf6fddac83d0f4315303d88273e607449d9ee2648e907ff20: Status 404 returned error can't find the container with id 46f852d8a18e29ddf6fddac83d0f4315303d88273e607449d9ee2648e907ff20
Jul 17 05:33:42.394899 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.394839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:42.395077 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:42.394909    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:42.395077 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:42.394951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:42.395077 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:42.394976    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls podName:a1df72a6-3180-44b4-a30d-a3e1917adccb nodeName:}" failed. No retries permitted until 2026-07-17 05:33:43.394956047 +0000 UTC m=+48.173438985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-74mbk" (UID: "a1df72a6-3180-44b4-a30d-a3e1917adccb") : secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:42.395077 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:42.395020    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 05:33:42.395077 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:42.395060    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls podName:a1935ff2-b7b3-4307-bce9-aa549479b4c9 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:43.395050147 +0000 UTC m=+48.173533085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4stv9" (UID: "a1935ff2-b7b3-4307-bce9-aa549479b4c9") : secret "samples-operator-tls" not found
Jul 17 05:33:43.067671 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:43.067572    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf" event={"ID":"755460e4-929f-4f3f-9b39-da00e1e93048","Type":"ContainerStarted","Data":"08af66b35c7d12e9ff0b5d019fb0c4d32010a0556b7f2aa43cda859c2d4b59b3"}
Jul 17 05:33:43.069894 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:43.069806    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw" event={"ID":"7f766e54-ad49-4a16-a8a2-71e727dbeab2","Type":"ContainerStarted","Data":"46f852d8a18e29ddf6fddac83d0f4315303d88273e607449d9ee2648e907ff20"}
Jul 17 05:33:43.071680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:43.071647    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-czqrr" event={"ID":"34afd671-dce9-4fce-8fa0-c265ca199e21","Type":"ContainerStarted","Data":"4ece2aa3f7339ca5bb8bfdfc2c55a8766d80afc4f474ebf70de611c37679f8c7"}
Jul 17 05:33:43.406267 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:43.406169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:43.406267 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:43.406264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:43.406484 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:43.406416    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 05:33:43.406554 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:43.406489    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls podName:a1935ff2-b7b3-4307-bce9-aa549479b4c9 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:45.406467795 +0000 UTC m=+50.184950733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4stv9" (UID: "a1935ff2-b7b3-4307-bce9-aa549479b4c9") : secret "samples-operator-tls" not found
Jul 17 05:33:43.406554 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:43.406502    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:43.406659 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:43.406577    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls podName:a1df72a6-3180-44b4-a30d-a3e1917adccb nodeName:}" failed. No retries permitted until 2026-07-17 05:33:45.406558828 +0000 UTC m=+50.185041772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-74mbk" (UID: "a1df72a6-3180-44b4-a30d-a3e1917adccb") : secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:45.079937 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.079834    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw" event={"ID":"7f766e54-ad49-4a16-a8a2-71e727dbeab2","Type":"ContainerStarted","Data":"40a1616a4948ceebf341165364a0a666dae4c8bf77a7381ea30bfafd3fae4c02"}
Jul 17 05:33:45.082133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.082101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-czqrr" event={"ID":"34afd671-dce9-4fce-8fa0-c265ca199e21","Type":"ContainerStarted","Data":"5ad391cb8083c284ac41b91461faacc186ecc69e09ece3771bd7e482fc9387fc"}
Jul 17 05:33:45.097202 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.097141    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7bwdw" podStartSLOduration=1.621588509 podStartE2EDuration="4.097124037s" totalImagesPullingTime="2.475535528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:45.095592758 +0000 UTC m=+49.874075719" watchObservedRunningTime="2026-07-17 05:33:45.097124037 +0000 UTC m=+49.875606999"
Jul 17 05:33:45.112778 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.112688    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-czqrr" podStartSLOduration=1.5710592330000002 podStartE2EDuration="4.112664316s" totalImagesPullingTime="2.541605083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:45.111909652 +0000 UTC m=+49.890392612" watchObservedRunningTime="2026-07-17 05:33:45.112664316 +0000 UTC m=+49.891147277"
Jul 17 05:33:45.170182 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.170081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:45.227659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.227622    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:45.426248 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.426149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:45.426248 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.426224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:45.426506 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:45.426348    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:45.426506 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:45.426366    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 05:33:45.426506 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:45.426434    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls podName:a1935ff2-b7b3-4307-bce9-aa549479b4c9 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:49.426414273 +0000 UTC m=+54.204897223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4stv9" (UID: "a1935ff2-b7b3-4307-bce9-aa549479b4c9") : secret "samples-operator-tls" not found
Jul 17 05:33:45.426506 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:45.426454    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls podName:a1df72a6-3180-44b4-a30d-a3e1917adccb nodeName:}" failed. No retries permitted until 2026-07-17 05:33:49.426445219 +0000 UTC m=+54.204928158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-74mbk" (UID: "a1df72a6-3180-44b4-a30d-a3e1917adccb") : secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:45.702682 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.702551    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:45.800368 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.800328    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:45.935785 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.935736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:45.983409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:45.983373    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:46.412973 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.412926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:46.467598 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.467557    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x6h6s"]
Jul 17 05:33:46.482059 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.481083    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.482483 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.482410    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x6h6s"]
Jul 17 05:33:46.484320 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.484297    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tk2pd\""
Jul 17 05:33:46.484813 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.484794    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 05:33:46.485045 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.484794    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 05:33:46.488242 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.488162    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:46.550983 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.550943    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:46.636963 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.636916    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm7ns\" (UniqueName: \"kubernetes.io/projected/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-api-access-gm7ns\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.637180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.636995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.637180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.637028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d502e8a-66f0-460d-9b74-b86338a4e52a-data-volume\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.637180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.637056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.637180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.637097    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2d502e8a-66f0-460d-9b74-b86338a4e52a-crio-socket\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.658516 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.658472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:46.737584 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.737540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm7ns\" (UniqueName: \"kubernetes.io/projected/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-api-access-gm7ns\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.737603    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.737648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d502e8a-66f0-460d-9b74-b86338a4e52a-data-volume\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.737680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:46.737704    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.737731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2d502e8a-66f0-460d-9b74-b86338a4e52a-crio-socket\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:46.737780    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:47.237760195 +0000 UTC m=+52.016243138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:46.737820    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:33:46.737919 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:46.737918    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:47.237899581 +0000 UTC m=+52.016382522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : configmap "kube-rbac-proxy" not found
Jul 17 05:33:46.738300 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.738039    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2d502e8a-66f0-460d-9b74-b86338a4e52a-crio-socket\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.738300 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.738138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d502e8a-66f0-460d-9b74-b86338a4e52a-data-volume\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.748799 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.748762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:46.750675 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.750648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm7ns\" (UniqueName: \"kubernetes.io/projected/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-api-access-gm7ns\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:46.810731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:46.810698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:47.088640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:47.088552    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf" event={"ID":"755460e4-929f-4f3f-9b39-da00e1e93048","Type":"ContainerStarted","Data":"0bd1317bc5bdcb2084c1ca9073ed5e088752d25d309f118c7655bfe06909ffab"}
Jul 17 05:33:47.102935 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:47.102884    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nvnxf" podStartSLOduration=2.148995432 podStartE2EDuration="6.102849968s" totalImagesPullingTime="3.953854536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:47.102060749 +0000 UTC m=+51.880543711" watchObservedRunningTime="2026-07-17 05:33:47.102849968 +0000 UTC m=+51.881332924"
Jul 17 05:33:47.242136 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:47.242089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:47.242346 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.242215    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:47.242346 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:47.242259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:47.242346 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.242276    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:48.242257855 +0000 UTC m=+53.020740808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:47.242346 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.242310    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:33:47.242346 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.242345    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:48.242334711 +0000 UTC m=+53.020817662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : configmap "kube-rbac-proxy" not found
Jul 17 05:33:47.343927 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:47.343826    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:33:47.344082 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.343973    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:33:47.344082 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.344060    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls podName:53c331fb-d346-4c18-b9eb-d0b8a8825437 nodeName:}" failed. No retries permitted until 2026-07-17 05:34:03.344043319 +0000 UTC m=+68.122526256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls") pod "dns-default-9m5m5" (UID: "53c331fb-d346-4c18-b9eb-d0b8a8825437") : secret "dns-default-metrics-tls" not found
Jul 17 05:33:47.444539 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:47.444500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:33:47.444958 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.444654    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:33:47.444958 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.444673    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69787f6b6f-8vhgd: secret "image-registry-tls" not found
Jul 17 05:33:47.444958 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:47.444738    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls podName:20673ee8-572c-4ad1-8a61-27f45fdf36bf nodeName:}" failed. No retries permitted until 2026-07-17 05:34:03.444719664 +0000 UTC m=+68.223202611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls") pod "image-registry-69787f6b6f-8vhgd" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf") : secret "image-registry-tls" not found
Jul 17 05:33:48.115512 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:48.115472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:48.162781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:48.162752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:48.252098 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:48.252052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:48.252285 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:48.252200    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:48.252285 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:48.252234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:48.252285 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:48.252272    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:50.252253035 +0000 UTC m=+55.030735989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:48.252454 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:48.252292    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:33:48.252454 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:48.252329    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:50.2523184 +0000 UTC m=+55.030801355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : configmap "kube-rbac-proxy" not found
Jul 17 05:33:48.259024 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:48.258999    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4mb5k_b8649966-7fcf-46ad-af87-392af43bb3df/dns-node-resolver/0.log"
Jul 17 05:33:48.873134 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:48.873094    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:49.259088 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:49.259019    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-47vtj_0833fd9c-124f-4b64-9618-cac8678daad0/node-ca/0.log"
Jul 17 05:33:49.463238 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:49.463175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:49.463383 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:49.463270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:49.463383 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:49.463330    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:49.463548 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:49.463396    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls podName:a1df72a6-3180-44b4-a30d-a3e1917adccb nodeName:}" failed. No retries permitted until 2026-07-17 05:33:57.463380747 +0000 UTC m=+62.241863685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-74mbk" (UID: "a1df72a6-3180-44b4-a30d-a3e1917adccb") : secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:49.463548 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:49.463410    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 05:33:49.463548 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:49.463456    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls podName:a1935ff2-b7b3-4307-bce9-aa549479b4c9 nodeName:}" failed. No retries permitted until 2026-07-17 05:33:57.463443849 +0000 UTC m=+62.241926792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4stv9" (UID: "a1935ff2-b7b3-4307-bce9-aa549479b4c9") : secret "samples-operator-tls" not found
Jul 17 05:33:50.260942 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:50.260906    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzrw2_d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae/serve-healthcheck-canary/0.log"
Jul 17 05:33:50.269106 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:50.269073    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:50.269250 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:50.269179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:50.269250 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:50.269192    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:33:50.269250 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:50.269248    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:54.269234188 +0000 UTC m=+59.047717127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : configmap "kube-rbac-proxy" not found
Jul 17 05:33:50.269353 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:50.269280    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:50.269353 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:50.269317    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:33:54.269304916 +0000 UTC m=+59.047787872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:52.279349 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:52.279314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:52.630929 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:52.630827    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:52.730873 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:52.730825    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:53.077176 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:53.077130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:53.197380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:53.197343    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:53.292556 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:53.292515    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:53.392131 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:53.392033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:53.927411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:53.927372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:54.303116 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:54.303082    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:54.303616 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:54.303147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:33:54.303616 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:54.303242    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:54.303616 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:54.303314    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:34:02.303296952 +0000 UTC m=+67.081779890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:33:54.303616 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:54.303248    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:33:54.303616 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:54.303407    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:34:02.303389103 +0000 UTC m=+67.081872059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : configmap "kube-rbac-proxy" not found
Jul 17 05:33:54.726603 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:54.726566    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:54.936944 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:54.936913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:55.336945 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:55.336909    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:55.525455 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:55.525421    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:55.581688 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:55.581657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:55.945352 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:55.945317    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:56.324708 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:56.324660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:56.431521 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:56.431478    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:56.731912 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:56.731879    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:56.931074 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:56.931031    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:57.018022 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.017941    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4l8jz"
Jul 17 05:33:57.327965 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.327868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:57.385176 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.385141    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:57.530285 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.530243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:33:57.530737 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.530329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:57.530737 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:57.530367    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:57.530737 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:33:57.530434    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls podName:a1df72a6-3180-44b4-a30d-a3e1917adccb nodeName:}" failed. No retries permitted until 2026-07-17 05:34:13.530419896 +0000 UTC m=+78.308902834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-74mbk" (UID: "a1df72a6-3180-44b4-a30d-a3e1917adccb") : secret "cluster-monitoring-operator-tls" not found
Jul 17 05:33:57.531381 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.531352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:57.533046 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.533023    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1935ff2-b7b3-4307-bce9-aa549479b4c9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4stv9\" (UID: \"a1935ff2-b7b3-4307-bce9-aa549479b4c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:57.622897 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.622787    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p6skv\""
Jul 17 05:33:57.631088 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.631062    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"
Jul 17 05:33:57.792670 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.792634    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9"]
Jul 17 05:33:57.924436 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:57.924344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:58.120616 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:58.120582    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9" event={"ID":"a1935ff2-b7b3-4307-bce9-aa549479b4c9","Type":"ContainerStarted","Data":"e9be4b56840e13b1136efbc9c631be7f77298c0659afd8ddcaa50d28b7340513"}
Jul 17 05:33:58.724790 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:58.724744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:58.826697 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:58.826650    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:58.957451 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:58.957409    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:59.330787 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:59.330751    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:59.541930 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:59.541889    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:33:59.973560 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:33:59.973528    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:00.128162 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.128126    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9" event={"ID":"a1935ff2-b7b3-4307-bce9-aa549479b4c9","Type":"ContainerStarted","Data":"3cc6a0369624e464024604f71cde513008a208f675101e95819d097dac324bf8"}
Jul 17 05:34:00.128367 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.128180    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9" event={"ID":"a1935ff2-b7b3-4307-bce9-aa549479b4c9","Type":"ContainerStarted","Data":"e712aa720087090c56ae7ca33d2fbaf306bea66738f1f60012631afff1df9e6d"}
Jul 17 05:34:00.146622 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.146561    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4stv9" podStartSLOduration=17.298555197 podStartE2EDuration="19.146544765s" totalImagesPullingTime="1.847989568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:00.145323257 +0000 UTC m=+64.923806216" watchObservedRunningTime="2026-07-17 05:34:00.146544765 +0000 UTC m=+64.925027724"
Jul 17 05:34:00.456467 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.456366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:34:00.459054 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.459032    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 05:34:00.469830 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.469795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93a50d18-6fa3-4037-8174-4ff5a09f6fb8-original-pull-secret\") pod \"global-pull-secret-syncer-7gz6h\" (UID: \"93a50d18-6fa3-4037-8174-4ff5a09f6fb8\") " pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:34:00.525688 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.525650    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:00.557073 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.557032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:34:00.559618 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.559596    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 05:34:00.569647 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.569613    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a08ba58-2146-46da-bb66-1132a32b2419-metrics-certs\") pod \"network-metrics-daemon-rk2tl\" (UID: \"6a08ba58-2146-46da-bb66-1132a32b2419\") " pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:34:00.657443 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.657408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:34:00.658890 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.658849    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44td2\""
Jul 17 05:34:00.660090 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.660064    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs6js\" (UniqueName: \"kubernetes.io/projected/0ac4b401-3f2b-4b26-ac52-0d87aa4f845c-kube-api-access-cs6js\") pod \"network-check-target-mt2jm\" (UID: \"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c\") " pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:34:00.662042 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.662024    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gz6h"
Jul 17 05:34:00.667105 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.667079    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rk2tl"
Jul 17 05:34:00.669457 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.669432    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x8c5l\""
Jul 17 05:34:00.677452 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.677420    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:34:00.805645 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.805603    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rk2tl"]
Jul 17 05:34:00.811955 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:00.811919    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a08ba58_2146_46da_bb66_1132a32b2419.slice/crio-9a21a50734d70e694e55993d67784f1af1e054d5bfd415e1c65eb4dc36c5d2a8 WatchSource:0}: Error finding container 9a21a50734d70e694e55993d67784f1af1e054d5bfd415e1c65eb4dc36c5d2a8: Status 404 returned error can't find the container with id 9a21a50734d70e694e55993d67784f1af1e054d5bfd415e1c65eb4dc36c5d2a8
Jul 17 05:34:00.822986 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.822954    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7gz6h"]
Jul 17 05:34:00.825387 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:00.825356    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a50d18_6fa3_4037_8174_4ff5a09f6fb8.slice/crio-603d602b4b5b89592e4c95b385a5c9f10852acaa2a8c2501f4593b09c9462fc3 WatchSource:0}: Error finding container 603d602b4b5b89592e4c95b385a5c9f10852acaa2a8c2501f4593b09c9462fc3: Status 404 returned error can't find the container with id 603d602b4b5b89592e4c95b385a5c9f10852acaa2a8c2501f4593b09c9462fc3
Jul 17 05:34:00.843420 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.843384    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mt2jm"]
Jul 17 05:34:00.846605 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:00.846575    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac4b401_3f2b_4b26_ac52_0d87aa4f845c.slice/crio-c67af6d14fe70e5eec624c6482079cb8f429c144febbcdda580ef654b491be3a WatchSource:0}: Error finding container c67af6d14fe70e5eec624c6482079cb8f429c144febbcdda580ef654b491be3a: Status 404 returned error can't find the container with id c67af6d14fe70e5eec624c6482079cb8f429c144febbcdda580ef654b491be3a
Jul 17 05:34:00.928796 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:00.928754    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:01.132921 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.132879    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rk2tl" event={"ID":"6a08ba58-2146-46da-bb66-1132a32b2419","Type":"ContainerStarted","Data":"9a21a50734d70e694e55993d67784f1af1e054d5bfd415e1c65eb4dc36c5d2a8"}
Jul 17 05:34:01.134111 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.134077    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7gz6h" event={"ID":"93a50d18-6fa3-4037-8174-4ff5a09f6fb8","Type":"ContainerStarted","Data":"603d602b4b5b89592e4c95b385a5c9f10852acaa2a8c2501f4593b09c9462fc3"}
Jul 17 05:34:01.135520 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.135490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mt2jm" event={"ID":"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c","Type":"ContainerStarted","Data":"e66dc6cfe8cfdb4a471434fbb9fe84176fe460bdf51de6a2d1745a639f437cc8"}
Jul 17 05:34:01.135646 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.135541    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mt2jm" event={"ID":"0ac4b401-3f2b-4b26-ac52-0d87aa4f845c","Type":"ContainerStarted","Data":"c67af6d14fe70e5eec624c6482079cb8f429c144febbcdda580ef654b491be3a"}
Jul 17 05:34:01.150290 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.150238    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mt2jm" podStartSLOduration=65.150220851 podStartE2EDuration="1m5.150220851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:01.149467321 +0000 UTC m=+65.927950283" watchObservedRunningTime="2026-07-17 05:34:01.150220851 +0000 UTC m=+65.928703802"
Jul 17 05:34:01.326940 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.326910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:01.748728 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.748689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:01.849893 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:01.849824    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:02.133330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.133293    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:02.141794 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.141750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rk2tl" event={"ID":"6a08ba58-2146-46da-bb66-1132a32b2419","Type":"ContainerStarted","Data":"945bf2867c445b6349ce2604cb50ba725616eff0523952ba145b8cc12e7d23e2"}
Jul 17 05:34:02.142016 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.141873    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rk2tl" event={"ID":"6a08ba58-2146-46da-bb66-1132a32b2419","Type":"ContainerStarted","Data":"2912d3d11b793dd20c4829b78bb414d7609f8a5041fcf03cb6fa9f5c17ad88b7"}
Jul 17 05:34:02.142016 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.141909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:34:02.158581 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.158502    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rk2tl" podStartSLOduration=65.103044228 podStartE2EDuration="1m6.158483097s" totalImagesPullingTime="1.055438869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:02.157820347 +0000 UTC m=+66.936303307" watchObservedRunningTime="2026-07-17 05:34:02.158483097 +0000 UTC m=+66.936966056"
Jul 17 05:34:02.340579 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.340495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:02.371595 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.371557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:34:02.371769 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.371600    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:34:02.371769 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:02.371705    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:34:02.371838 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:02.371818    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls podName:2d502e8a-66f0-460d-9b74-b86338a4e52a nodeName:}" failed. No retries permitted until 2026-07-17 05:34:18.371800075 +0000 UTC m=+83.150283012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6h6s" (UID: "2d502e8a-66f0-460d-9b74-b86338a4e52a") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:34:02.372134 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.372118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2d502e8a-66f0-460d-9b74-b86338a4e52a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:34:02.725543 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:02.725500    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:03.136475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.136385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:03.323875 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.323806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:03.379964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.379924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:34:03.382490 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.382460    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53c331fb-d346-4c18-b9eb-d0b8a8825437-metrics-tls\") pod \"dns-default-9m5m5\" (UID: \"53c331fb-d346-4c18-b9eb-d0b8a8825437\") " pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:34:03.480435 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.480395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:03.483135 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.483108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"image-registry-69787f6b6f-8vhgd\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") " pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:03.583900 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.583851    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mxhcm\""
Jul 17 05:34:03.591993 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.591960    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:03.603236 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.603208    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qq4z7\""
Jul 17 05:34:03.611560 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:03.611518    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:34:04.670085 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:04.670025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:04.726141 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:04.726107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:05.071922 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:05.071887    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9m5m5"]
Jul 17 05:34:05.076118 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:05.076084    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c331fb_d346_4c18_b9eb_d0b8a8825437.slice/crio-63879c73a391ef2f8dbe4276840e979c6d124c478b40322fef09f772844cf318 WatchSource:0}: Error finding container 63879c73a391ef2f8dbe4276840e979c6d124c478b40322fef09f772844cf318: Status 404 returned error can't find the container with id 63879c73a391ef2f8dbe4276840e979c6d124c478b40322fef09f772844cf318
Jul 17 05:34:05.089592 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:05.089559    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69787f6b6f-8vhgd"]
Jul 17 05:34:05.092519 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:05.092491    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20673ee8_572c_4ad1_8a61_27f45fdf36bf.slice/crio-040cadc6cc29d36bf516cebf6715272643a4b444dbf50d724f1d54eaed8f08ea WatchSource:0}: Error finding container 040cadc6cc29d36bf516cebf6715272643a4b444dbf50d724f1d54eaed8f08ea: Status 404 returned error can't find the container with id 040cadc6cc29d36bf516cebf6715272643a4b444dbf50d724f1d54eaed8f08ea
Jul 17 05:34:05.154508 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:05.154471    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m5m5" event={"ID":"53c331fb-d346-4c18-b9eb-d0b8a8825437","Type":"ContainerStarted","Data":"63879c73a391ef2f8dbe4276840e979c6d124c478b40322fef09f772844cf318"}
Jul 17 05:34:05.155582 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:05.155555    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" event={"ID":"20673ee8-572c-4ad1-8a61-27f45fdf36bf","Type":"ContainerStarted","Data":"040cadc6cc29d36bf516cebf6715272643a4b444dbf50d724f1d54eaed8f08ea"}
Jul 17 05:34:06.160210 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:06.160168    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7gz6h" event={"ID":"93a50d18-6fa3-4037-8174-4ff5a09f6fb8","Type":"ContainerStarted","Data":"937ed0331ef74d20877a4807685f2a689882653c63e396cbe4a0c8b97336c91d"}
Jul 17 05:34:06.161706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:06.161675    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" event={"ID":"20673ee8-572c-4ad1-8a61-27f45fdf36bf","Type":"ContainerStarted","Data":"0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412"}
Jul 17 05:34:06.161797 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:06.161744    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:06.175042 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:06.174985    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7gz6h" podStartSLOduration=65.72122095 podStartE2EDuration="1m10.174967523s" totalImagesPullingTime="4.453746573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:06.173089729 +0000 UTC m=+70.951572689" watchObservedRunningTime="2026-07-17 05:34:06.174967523 +0000 UTC m=+70.953450483"
Jul 17 05:34:06.192477 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:06.192422    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" podStartSLOduration=70.192404082 podStartE2EDuration="1m10.192404082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:06.191550152 +0000 UTC m=+70.970033113" watchObservedRunningTime="2026-07-17 05:34:06.192404082 +0000 UTC m=+70.970887104"
Jul 17 05:34:08.170609 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.170573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m5m5" event={"ID":"53c331fb-d346-4c18-b9eb-d0b8a8825437","Type":"ContainerStarted","Data":"56774a72afa44ad96bcc429d30cd9235ecd83f5ac1f0e8f7a75ea170880d0ce7"}
Jul 17 05:34:08.171218 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.170636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m5m5" event={"ID":"53c331fb-d346-4c18-b9eb-d0b8a8825437","Type":"ContainerStarted","Data":"ceaeb3bca50191deb491a4c57bf33b3599a4845c574220547af217065d65ab97"}
Jul 17 05:34:08.171218 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.170693    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:34:08.196928 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.196889    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"]
Jul 17 05:34:08.199205 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.199177    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.202250 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.202222    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 05:34:08.202401 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.202261    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 05:34:08.202401 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.202222    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 05:34:08.202917 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.202899    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 17 05:34:08.224787 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.224723    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9m5m5" podStartSLOduration=35.240604293 podStartE2EDuration="37.224698701s" totalImagesPullingTime="1.984094408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:08.219357509 +0000 UTC m=+72.997840471" watchObservedRunningTime="2026-07-17 05:34:08.224698701 +0000 UTC m=+73.003181670"
Jul 17 05:34:08.226075 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.226044    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"]
Jul 17 05:34:08.323691 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.323648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd9fea2-b2d3-48b0-acf7-393c05931f1f-tmp\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.323691 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.323691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/afd9fea2-b2d3-48b0-acf7-393c05931f1f-klusterlet-config\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.323952 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.323815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9wtq\" (UniqueName: \"kubernetes.io/projected/afd9fea2-b2d3-48b0-acf7-393c05931f1f-kube-api-access-f9wtq\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.362030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.361999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:08.424313 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.424209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9wtq\" (UniqueName: \"kubernetes.io/projected/afd9fea2-b2d3-48b0-acf7-393c05931f1f-kube-api-access-f9wtq\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.424313 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.424258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd9fea2-b2d3-48b0-acf7-393c05931f1f-tmp\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.424313 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.424275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/afd9fea2-b2d3-48b0-acf7-393c05931f1f-klusterlet-config\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.424702 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.424677    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd9fea2-b2d3-48b0-acf7-393c05931f1f-tmp\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.426973 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.426952    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/afd9fea2-b2d3-48b0-acf7-393c05931f1f-klusterlet-config\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.433896 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.433846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9wtq\" (UniqueName: \"kubernetes.io/projected/afd9fea2-b2d3-48b0-acf7-393c05931f1f-kube-api-access-f9wtq\") pod \"klusterlet-addon-workmgr-65cf5459d9-v9cff\" (UID: \"afd9fea2-b2d3-48b0-acf7-393c05931f1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.455357 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.455322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:08.509266 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.509224    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:08.568012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.567980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:08.648151 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.648010    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"]
Jul 17 05:34:08.651137 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:08.651110    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd9fea2_b2d3_48b0_acf7_393c05931f1f.slice/crio-b1562850acb1ecd8b745bb09c1f5f7af2fe700bb6e8f57c2d396d49cfccd6a65 WatchSource:0}: Error finding container b1562850acb1ecd8b745bb09c1f5f7af2fe700bb6e8f57c2d396d49cfccd6a65: Status 404 returned error can't find the container with id b1562850acb1ecd8b745bb09c1f5f7af2fe700bb6e8f57c2d396d49cfccd6a65
Jul 17 05:34:08.657101 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.657063    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:08.819071 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.819038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:08.914545 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:08.914513    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:09.174845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:09.174749    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff" event={"ID":"afd9fea2-b2d3-48b0-acf7-393c05931f1f","Type":"ContainerStarted","Data":"b1562850acb1ecd8b745bb09c1f5f7af2fe700bb6e8f57c2d396d49cfccd6a65"}
Jul 17 05:34:10.441404 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:10.441268    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:10.538215 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:10.538177    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:10.605193 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:10.605152    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:13.190541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.190503    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff" event={"ID":"afd9fea2-b2d3-48b0-acf7-393c05931f1f","Type":"ContainerStarted","Data":"18fa94b1da0d0e66401f206211a10a20c635b2eb2278af50362d315af63a3dd2"}
Jul 17 05:34:13.191003 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.190679    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:13.192368 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.192344    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff"
Jul 17 05:34:13.206884 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.206803    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65cf5459d9-v9cff" podStartSLOduration=1.658328176 podStartE2EDuration="5.206786288s" totalImagesPullingTime="3.548458112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:13.205655752 +0000 UTC m=+77.984138713" watchObservedRunningTime="2026-07-17 05:34:13.206786288 +0000 UTC m=+77.985269248"
Jul 17 05:34:13.566517 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.566476    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:34:13.568992 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.568962    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1df72a6-3180-44b4-a30d-a3e1917adccb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-74mbk\" (UID: \"a1df72a6-3180-44b4-a30d-a3e1917adccb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:34:13.799658 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.799623    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kdnlm\""
Jul 17 05:34:13.807635 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.807606    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"
Jul 17 05:34:13.943423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:13.943389    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk"]
Jul 17 05:34:13.946589 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:13.946546    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1df72a6_3180_44b4_a30d_a3e1917adccb.slice/crio-6cdbda6c1440874757ac33dea2eaf20566a0b842609029433192fcecce60656c WatchSource:0}: Error finding container 6cdbda6c1440874757ac33dea2eaf20566a0b842609029433192fcecce60656c: Status 404 returned error can't find the container with id 6cdbda6c1440874757ac33dea2eaf20566a0b842609029433192fcecce60656c
Jul 17 05:34:14.195658 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:14.195566    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk" event={"ID":"a1df72a6-3180-44b4-a30d-a3e1917adccb","Type":"ContainerStarted","Data":"6cdbda6c1440874757ac33dea2eaf20566a0b842609029433192fcecce60656c"}
Jul 17 05:34:16.011512 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.011477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:16.169891 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.169843    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"]
Jul 17 05:34:16.170140 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.170076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:16.172895 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.172877    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:16.175285 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.175260    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 05:34:16.175390 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.175337    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qj4hv\""
Jul 17 05:34:16.181425 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.181395    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"]
Jul 17 05:34:16.204095 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.204052    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk" event={"ID":"a1df72a6-3180-44b4-a30d-a3e1917adccb","Type":"ContainerStarted","Data":"281ac6423b24afa1387def22df89f7de13319159e5443ecfc9f98ecc81723848"}
Jul 17 05:34:16.229869 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.229802    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-74mbk" podStartSLOduration=33.494697416 podStartE2EDuration="35.229782786s" totalImagesPullingTime="1.73508537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:16.228973359 +0000 UTC m=+81.007456319" watchObservedRunningTime="2026-07-17 05:34:16.229782786 +0000 UTC m=+81.008265746"
Jul 17 05:34:16.289655 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.289552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a960fc61-ed51-42d3-8478-b0e5fbe27dd3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rvj4z\" (UID: \"a960fc61-ed51-42d3-8478-b0e5fbe27dd3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:16.390021 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.389956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a960fc61-ed51-42d3-8478-b0e5fbe27dd3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rvj4z\" (UID: \"a960fc61-ed51-42d3-8478-b0e5fbe27dd3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:16.390246 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:16.390155    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 05:34:16.390323 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:16.390248    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a960fc61-ed51-42d3-8478-b0e5fbe27dd3-tls-certificates podName:a960fc61-ed51-42d3-8478-b0e5fbe27dd3 nodeName:}" failed. No retries permitted until 2026-07-17 05:34:16.890223926 +0000 UTC m=+81.668706864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a960fc61-ed51-42d3-8478-b0e5fbe27dd3-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-rvj4z" (UID: "a960fc61-ed51-42d3-8478-b0e5fbe27dd3") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 05:34:16.510648 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.510613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:16.611051 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.610951    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:16.663717 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.663677    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:16.893803 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.893686    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a960fc61-ed51-42d3-8478-b0e5fbe27dd3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rvj4z\" (UID: \"a960fc61-ed51-42d3-8478-b0e5fbe27dd3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:16.896346 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:16.896320    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a960fc61-ed51-42d3-8478-b0e5fbe27dd3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rvj4z\" (UID: \"a960fc61-ed51-42d3-8478-b0e5fbe27dd3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:17.085193 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:17.085149    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:17.219043 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:17.219008    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"]
Jul 17 05:34:17.222200 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:17.222174    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda960fc61_ed51_42d3_8478_b0e5fbe27dd3.slice/crio-c8b0107444b84e0fc2825e204bce956272f13713e58b3684c380761011f2b27a WatchSource:0}: Error finding container c8b0107444b84e0fc2825e204bce956272f13713e58b3684c380761011f2b27a: Status 404 returned error can't find the container with id c8b0107444b84e0fc2825e204bce956272f13713e58b3684c380761011f2b27a
Jul 17 05:34:18.177468 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.177433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9m5m5"
Jul 17 05:34:18.212403 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.212364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z" event={"ID":"a960fc61-ed51-42d3-8478-b0e5fbe27dd3","Type":"ContainerStarted","Data":"c8b0107444b84e0fc2825e204bce956272f13713e58b3684c380761011f2b27a"}
Jul 17 05:34:18.407402 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.407363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:34:18.409721 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.409698    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2d502e8a-66f0-460d-9b74-b86338a4e52a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6h6s\" (UID: \"2d502e8a-66f0-460d-9b74-b86338a4e52a\") " pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:34:18.597309 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.597279    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tk2pd\""
Jul 17 05:34:18.605337 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.605061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x6h6s"
Jul 17 05:34:18.769925 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:18.769802    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x6h6s"]
Jul 17 05:34:18.772506 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:18.772478    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d502e8a_66f0_460d_9b74_b86338a4e52a.slice/crio-4ce22e71cd369845eb8e94dd376c6e615f158be514ca3ee3c84b3db271b1fa07 WatchSource:0}: Error finding container 4ce22e71cd369845eb8e94dd376c6e615f158be514ca3ee3c84b3db271b1fa07: Status 404 returned error can't find the container with id 4ce22e71cd369845eb8e94dd376c6e615f158be514ca3ee3c84b3db271b1fa07
Jul 17 05:34:19.217836 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.217791    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6h6s" event={"ID":"2d502e8a-66f0-460d-9b74-b86338a4e52a","Type":"ContainerStarted","Data":"4ce22e71cd369845eb8e94dd376c6e615f158be514ca3ee3c84b3db271b1fa07"}
Jul 17 05:34:19.219740 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.219708    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z" event={"ID":"a960fc61-ed51-42d3-8478-b0e5fbe27dd3","Type":"ContainerStarted","Data":"36730c2c861517425d87f490d621df8c5c8fed3889f491c19fe0cd3bddfa6e99"}
Jul 17 05:34:19.219956 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.219896    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:19.225721 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.225690    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z"
Jul 17 05:34:19.237437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.237383    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rvj4z" podStartSLOduration=1.890598591 podStartE2EDuration="3.237363046s" totalImagesPullingTime="1.346764455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:34:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:19.236364554 +0000 UTC m=+84.014847517" watchObservedRunningTime="2026-07-17 05:34:19.237363046 +0000 UTC m=+84.015846023"
Jul 17 05:34:19.661475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.661388    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:19.723322 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:19.723270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:20.226729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.226691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:20.260737 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.260704    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"]
Jul 17 05:34:20.262890 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.262852    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.265229 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.265201    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 17 05:34:20.265380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.265265    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 17 05:34:20.265380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.265300    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bv9v6\""
Jul 17 05:34:20.266369 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.266342    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 05:34:20.274659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.274629    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"]
Jul 17 05:34:20.294172 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.294139    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:20.322783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.322750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.323006 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.322790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdcjg\" (UniqueName: \"kubernetes.io/projected/9b79b45e-5bcb-43ec-b855-a2e6168620d5-kube-api-access-pdcjg\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.323006 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.322823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.323006 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.322877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b79b45e-5bcb-43ec-b855-a2e6168620d5-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.424150 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.424110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.424329 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.424156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdcjg\" (UniqueName: \"kubernetes.io/projected/9b79b45e-5bcb-43ec-b855-a2e6168620d5-kube-api-access-pdcjg\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.424329 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.424186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.424329 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.424225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b79b45e-5bcb-43ec-b855-a2e6168620d5-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.424595 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:20.424573    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 17 05:34:20.424659 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:20.424645    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-tls podName:9b79b45e-5bcb-43ec-b855-a2e6168620d5 nodeName:}" failed. No retries permitted until 2026-07-17 05:34:20.924622326 +0000 UTC m=+85.703105269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-b7l6l" (UID: "9b79b45e-5bcb-43ec-b855-a2e6168620d5") : secret "prometheus-operator-tls" not found
Jul 17 05:34:20.425627 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.425472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b79b45e-5bcb-43ec-b855-a2e6168620d5-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.429384 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.428765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.435190 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.435130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdcjg\" (UniqueName: \"kubernetes.io/projected/9b79b45e-5bcb-43ec-b855-a2e6168620d5-kube-api-access-pdcjg\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.502909 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.502790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:20.929431 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.929326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:20.932208 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:20.932178    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b79b45e-5bcb-43ec-b855-a2e6168620d5-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-b7l6l\" (UID: \"9b79b45e-5bcb-43ec-b855-a2e6168620d5\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:21.176021 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:21.175980    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"
Jul 17 05:34:23.119947 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:23.119905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:23.184505 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:23.184469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:23.597789 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:23.597748    2559 patch_prober.go:28] interesting pod/image-registry-69787f6b6f-8vhgd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:34:23.597991 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:23.597814    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:34:24.001177 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:24.001105    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l"]
Jul 17 05:34:24.003593 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:24.003561    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b79b45e_5bcb_43ec_b855_a2e6168620d5.slice/crio-78bafad2d593c7bdf660b977a77fd31c468fd73b3b99bb4797a11bdc0c7edd4e WatchSource:0}: Error finding container 78bafad2d593c7bdf660b977a77fd31c468fd73b3b99bb4797a11bdc0c7edd4e: Status 404 returned error can't find the container with id 78bafad2d593c7bdf660b977a77fd31c468fd73b3b99bb4797a11bdc0c7edd4e
Jul 17 05:34:24.239509 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:24.239479    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6h6s" event={"ID":"2d502e8a-66f0-460d-9b74-b86338a4e52a","Type":"ContainerStarted","Data":"fc5fd014daf1623e89c350f32285d03a7fbe390f82c1449aeddc326f869728bb"}
Jul 17 05:34:24.240650 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:24.240618    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l" event={"ID":"9b79b45e-5bcb-43ec-b855-a2e6168620d5","Type":"ContainerStarted","Data":"78bafad2d593c7bdf660b977a77fd31c468fd73b3b99bb4797a11bdc0c7edd4e"}
Jul 17 05:34:25.690253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:25.690139    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:26.251985 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:26.251939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l" event={"ID":"9b79b45e-5bcb-43ec-b855-a2e6168620d5","Type":"ContainerStarted","Data":"959dc63dd46477a2ba9f5211ea200401df40c20b9e544f707c34e31439944f32"}
Jul 17 05:34:26.252187 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:26.252050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l" event={"ID":"9b79b45e-5bcb-43ec-b855-a2e6168620d5","Type":"ContainerStarted","Data":"e91154e280daeee7c32c35463de323f0ec72b821d215d15e3d569e4a5d80b982"}
Jul 17 05:34:26.273135 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:26.273076    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-b7l6l" podStartSLOduration=4.897565522 podStartE2EDuration="6.273057024s" totalImagesPullingTime="1.375491502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:26.271090247 +0000 UTC m=+91.049573208" watchObservedRunningTime="2026-07-17 05:34:26.273057024 +0000 UTC m=+91.051539983"
Jul 17 05:34:26.506513 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:26.506420    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:26.610879 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:26.610807    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:26.709403 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:26.709366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:27.171452 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:27.171419    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:27.313466 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:27.313397    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:27.415575 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:27.415535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:27.514459 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:27.514418    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.261984 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.261941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.263080 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.263047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6h6s" event={"ID":"2d502e8a-66f0-460d-9b74-b86338a4e52a","Type":"ContainerStarted","Data":"b4475bb6c99c61da950dd534908eb34b524614cb1b41ae449ff5af4ab248ad8b"}
Jul 17 05:34:28.368356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.368313    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.464917 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.464876    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.570484 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.570371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.670157 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.670119    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.687891 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.687838    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-glwxm"]
Jul 17 05:34:28.691672 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.691646    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.694430 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.694365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 05:34:28.694583 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.694560    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 05:34:28.694700 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.694664    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vr8h8\""
Jul 17 05:34:28.694700 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.694677    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 05:34:28.760703 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.760663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.793903 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.793865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-root\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.793922    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-metrics-client-ca\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.793964    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-sys\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.793985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl5wh\" (UniqueName: \"kubernetes.io/projected/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-kube-api-access-rl5wh\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.794013    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-textfile\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.794063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-tls\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.794110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-wtmp\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.794141    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.794450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.794168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-accelerators-collector-config\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.859969 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.859845    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:28.895012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.894970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-tls\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895041    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-wtmp\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895078    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-accelerators-collector-config\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-root\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895174    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-metrics-client-ca\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895223 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-sys\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl5wh\" (UniqueName: \"kubernetes.io/projected/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-kube-api-access-rl5wh\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-textfile\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-wtmp\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-root\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-sys\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.895791 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.895604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-textfile\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.896287 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.896241    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-metrics-client-ca\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.896429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.896403    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-accelerators-collector-config\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.897966 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.897940    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-tls\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.898108 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.897993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.905228 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.905183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl5wh\" (UniqueName: \"kubernetes.io/projected/f58dc1c9-0eb2-4631-8813-21fa11f0cf63-kube-api-access-rl5wh\") pod \"node-exporter-glwxm\" (UID: \"f58dc1c9-0eb2-4631-8813-21fa11f0cf63\") " pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:28.959941 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:28.959901    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.003431 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.003378    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-glwxm"
Jul 17 05:34:29.016583 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:29.016537    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58dc1c9_0eb2_4631_8813_21fa11f0cf63.slice/crio-a1d25f2aa240eb7cd1390878e8a840afb0aee206298e622b13aa4ec61330835e WatchSource:0}: Error finding container a1d25f2aa240eb7cd1390878e8a840afb0aee206298e622b13aa4ec61330835e: Status 404 returned error can't find the container with id a1d25f2aa240eb7cd1390878e8a840afb0aee206298e622b13aa4ec61330835e
Jul 17 05:34:29.059386 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.059339    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.200815 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.200695    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.268724 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.268684    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glwxm" event={"ID":"f58dc1c9-0eb2-4631-8813-21fa11f0cf63","Type":"ContainerStarted","Data":"a1d25f2aa240eb7cd1390878e8a840afb0aee206298e622b13aa4ec61330835e"}
Jul 17 05:34:29.335940 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.335895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.436823 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.436780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.538888 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.538823    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.639396 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.639323    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.742769 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.742731    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.855836 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.855744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:29.966320 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:29.966235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:30.061996 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:30.061952    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:30.155464 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:30.155384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:30.258571 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:30.258516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:30.556022 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:30.555980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:31.405712 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:31.405671    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69787f6b6f-8vhgd"]
Jul 17 05:34:31.499437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:31.499318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:31.601303 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:31.601261    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:31.702254 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:31.702194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:31.801128 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:31.801097    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:31.900330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:31.900291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:32.198405 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.198310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:32.266228 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.266195    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:32.288867 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.288815    2559 generic.go:356] "Generic (PLEG): container finished" podID="f58dc1c9-0eb2-4631-8813-21fa11f0cf63" containerID="62860745ad2e5776fcd38c5dd9ba6921f3a3891c75928c34002980242f9a644f" exitCode=0
Jul 17 05:34:32.289027 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.288902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glwxm" event={"ID":"f58dc1c9-0eb2-4631-8813-21fa11f0cf63","Type":"ContainerDied","Data":"62860745ad2e5776fcd38c5dd9ba6921f3a3891c75928c34002980242f9a644f"}
Jul 17 05:34:32.291335 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.291297    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6h6s" event={"ID":"2d502e8a-66f0-460d-9b74-b86338a4e52a","Type":"ContainerStarted","Data":"75263282a42faa9954ec197532f1c57373cc4fea66b3ad6210bd6f2a0a249880"}
Jul 17 05:34:32.331578 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.331514    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x6h6s" podStartSLOduration=34.012094841 podStartE2EDuration="46.331489964s" totalImagesPullingTime="12.319395123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:33:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:32.331027749 +0000 UTC m=+97.109510706" watchObservedRunningTime="2026-07-17 05:34:32.331489964 +0000 UTC m=+97.109972924"
Jul 17 05:34:32.341071 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.341022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:32.393897 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:32.393846    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:33.149009 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.148975    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mt2jm"
Jul 17 05:34:33.297313 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.297271    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glwxm" event={"ID":"f58dc1c9-0eb2-4631-8813-21fa11f0cf63","Type":"ContainerStarted","Data":"f3449423035592b70372e63367dae6279f2e308c9e5ec0c67c45c1fe579dc9a6"}
Jul 17 05:34:33.297486 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.297335    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glwxm" event={"ID":"f58dc1c9-0eb2-4631-8813-21fa11f0cf63","Type":"ContainerStarted","Data":"7456f878a516d574f7fa66df0fcaddb6a021d0da28b46c72c516f31800dcdb78"}
Jul 17 05:34:33.320072 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.320018    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-glwxm" podStartSLOduration=2.5437165459999997 podStartE2EDuration="5.32000014s" totalImagesPullingTime="2.707169456s" totalInitContainerRuntime="69.114138ms" isStatefulPod=true podCreationTimestamp="2026-07-17 05:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:33.31915657 +0000 UTC m=+98.097639557" watchObservedRunningTime="2026-07-17 05:34:33.32000014 +0000 UTC m=+98.098483099"
Jul 17 05:34:33.414258 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.414170    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"]
Jul 17 05:34:33.418221 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.418148    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:33.419542 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.419502    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:33.420662 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.420638    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 05:34:33.420777 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.420648    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l5fdr\""
Jul 17 05:34:33.432800 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.430924    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"]
Jul 17 05:34:33.494410 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.494374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:33.538398 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.538354    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49130648-2b3a-4e3b-afb2-934db8e9b287-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:33.538590 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.538464    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lwnp\" (UniqueName: \"kubernetes.io/projected/49130648-2b3a-4e3b-afb2-934db8e9b287-kube-api-access-5lwnp\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:33.639891 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.639821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49130648-2b3a-4e3b-afb2-934db8e9b287-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:33.640068 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.639938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lwnp\" (UniqueName: \"kubernetes.io/projected/49130648-2b3a-4e3b-afb2-934db8e9b287-kube-api-access-5lwnp\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:33.640068 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:33.639986    2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 17 05:34:33.640068 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:33.640058    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49130648-2b3a-4e3b-afb2-934db8e9b287-monitoring-plugin-cert podName:49130648-2b3a-4e3b-afb2-934db8e9b287 nodeName:}" failed. No retries permitted until 2026-07-17 05:34:34.140040692 +0000 UTC m=+98.918523630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/49130648-2b3a-4e3b-afb2-934db8e9b287-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-kn4z7" (UID: "49130648-2b3a-4e3b-afb2-934db8e9b287") : secret "monitoring-plugin-cert" not found
Jul 17 05:34:33.655773 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:33.655736    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lwnp\" (UniqueName: \"kubernetes.io/projected/49130648-2b3a-4e3b-afb2-934db8e9b287-kube-api-access-5lwnp\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:34.080166 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.080115    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:34.143985 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.143942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49130648-2b3a-4e3b-afb2-934db8e9b287-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:34.146500 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.146461    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49130648-2b3a-4e3b-afb2-934db8e9b287-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kn4z7\" (UID: \"49130648-2b3a-4e3b-afb2-934db8e9b287\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:34.328980 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.328942    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:34.478584 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.478549    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"]
Jul 17 05:34:34.482223 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:34.482193    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49130648_2b3a_4e3b_afb2_934db8e9b287.slice/crio-8b5708e552f48c83a902f2fc0160aeac534dedf5afe8ce5ead5a40cad04831ec WatchSource:0}: Error finding container 8b5708e552f48c83a902f2fc0160aeac534dedf5afe8ce5ead5a40cad04831ec: Status 404 returned error can't find the container with id 8b5708e552f48c83a902f2fc0160aeac534dedf5afe8ce5ead5a40cad04831ec
Jul 17 05:34:34.679835 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.679748    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:34.781638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.781599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:34.959452 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.957766    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:34.959704 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.959660    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:34:34.964332 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.964303    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:34.967933 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.967901    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 17 05:34:34.968094 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.967968    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4ug2f32of0sv9\""
Jul 17 05:34:34.968094 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.968055    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 17 05:34:34.968228 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.968062    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 17 05:34:34.968228 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.967912    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 17 05:34:34.968338 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.968290    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 05:34:34.968909 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.968887    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 17 05:34:34.969256 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969233    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 17 05:34:34.969356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969264    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 17 05:34:34.969356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969341    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 17 05:34:34.969500 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969434    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 17 05:34:34.969558 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969516    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 17 05:34:34.970461 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969827    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 17 05:34:34.970461 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969928    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 17 05:34:34.970461 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.969982    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xqbrz\""
Jul 17 05:34:34.971671 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.971639    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 17 05:34:34.973979 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.973782    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 17 05:34:34.980967 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:34.980930    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:34:35.051922 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.051881    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.051922 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.051934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.051976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052007    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052207    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052511 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052511 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052511 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052511 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052389    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052511 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052511 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052547    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4wdn\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-kube-api-access-d4wdn\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052629    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052671    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.052731 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.052705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153477 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4wdn\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-kube-api-access-d4wdn\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153560    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.153680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153840    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153914    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.153996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154234 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154234 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154234 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154234 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154234 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154474 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154474 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154408    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.154735 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.155412 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154990    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.155767 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.155737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.156808 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.156305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.156808 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.154237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.156808 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.156380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.156808 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.156421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.157475 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.157439    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.158144 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.158116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.158467 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.158441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.160964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.159493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.160964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.159493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.160964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.159933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.160964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.159967    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.160964 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.160526    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.161372 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.161116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.161372 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.161120    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.161560 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.161522    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.162017 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.161976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.164374 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.164324    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.166585 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.166563    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4wdn\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-kube-api-access-d4wdn\") pod \"prometheus-k8s-0\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.283619 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.283573    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:35.307253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.307187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7" event={"ID":"49130648-2b3a-4e3b-afb2-934db8e9b287","Type":"ContainerStarted","Data":"8b5708e552f48c83a902f2fc0160aeac534dedf5afe8ce5ead5a40cad04831ec"}
Jul 17 05:34:35.466640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:35.466258    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:34:35.469167 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:34:35.469137    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db49aa1_9f71_44c6_b68e_a371832e4b0b.slice/crio-2aeb4a4004195fae21af04b6f7510be881927bbd691e1441b9507a45094f82fc WatchSource:0}: Error finding container 2aeb4a4004195fae21af04b6f7510be881927bbd691e1441b9507a45094f82fc: Status 404 returned error can't find the container with id 2aeb4a4004195fae21af04b6f7510be881927bbd691e1441b9507a45094f82fc
Jul 17 05:34:36.313583 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.313538    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7" event={"ID":"49130648-2b3a-4e3b-afb2-934db8e9b287","Type":"ContainerStarted","Data":"537f9d6d8cb0bb5b46e9bdd6761c9c6bc523a3857c74985bd27e0df52d54a4c7"}
Jul 17 05:34:36.313793 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.313642    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:36.315135 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.315097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"2aeb4a4004195fae21af04b6f7510be881927bbd691e1441b9507a45094f82fc"}
Jul 17 05:34:36.320059 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.320032    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7"
Jul 17 05:34:36.321265 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.321234    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:36.330354 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.330296    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kn4z7" podStartSLOduration=1.9631444679999999 podStartE2EDuration="3.330273087s" totalImagesPullingTime="1.367128619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:36.32868311 +0000 UTC m=+101.107166072" watchObservedRunningTime="2026-07-17 05:34:36.330273087 +0000 UTC m=+101.108756048"
Jul 17 05:34:36.422371 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:36.422330    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:37.320384 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:37.320349    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7" exitCode=0
Jul 17 05:34:37.320851 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:37.320438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"}
Jul 17 05:34:41.338618 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:41.338581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"}
Jul 17 05:34:41.339019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:41.338651    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"}
Jul 17 05:34:43.349654 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.349620    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"}
Jul 17 05:34:43.350090 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.349706    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"}
Jul 17 05:34:43.350090 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.349748    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"}
Jul 17 05:34:43.350090 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.349780    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerStarted","Data":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"}
Jul 17 05:34:43.382544 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.382473    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.181940096 podStartE2EDuration="9.382446858s" totalImagesPullingTime="6.181047354s" totalInitContainerRuntime="19.459408ms" isStatefulPod=false podCreationTimestamp="2026-07-17 05:34:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:34:43.379768255 +0000 UTC m=+108.158251215" watchObservedRunningTime="2026-07-17 05:34:43.382446858 +0000 UTC m=+108.160929833"
Jul 17 05:34:43.754018 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.753977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:43.855058 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:43.855012    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:45.113485 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:45.113443    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:45.172720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:45.172680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:45.284730 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:45.284693    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:34:48.152306 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:48.152268    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:49.667892 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:49.667838    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:49.767988 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:49.767941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:53.550107 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:53.550045    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:53.601837 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:53.601802    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:54.750104 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:54.750065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:56.155374 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:56.155335    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:56.238245 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:56.238204    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:34:56.430210 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:56.430110    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerName="registry" containerID="cri-o://0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412" gracePeriod=30
Jul 17 05:34:57.267555 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.267508    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:57.346633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346537    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-bound-sa-token\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346587    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-image-registry-private-configuration\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346608    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20673ee8-572c-4ad1-8a61-27f45fdf36bf-ca-trust-extracted\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346962 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346644    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-installation-pull-secrets\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346962 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346678    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8kg7\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-kube-api-access-j8kg7\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346962 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346719    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-trusted-ca\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346962 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346756    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.346962 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.346773    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-certificates\") pod \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\" (UID: \"20673ee8-572c-4ad1-8a61-27f45fdf36bf\") "
Jul 17 05:34:57.347394 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.347298    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:34:57.347394 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.347350    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:34:57.349334 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.349297    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:34:57.349519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.349436    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-kube-api-access-j8kg7" (OuterVolumeSpecName: "kube-api-access-j8kg7") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "kube-api-access-j8kg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:34:57.349519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.349465    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:34:57.349519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.349491    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:34:57.349641 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.349613    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:34:57.356050 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.356017    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20673ee8-572c-4ad1-8a61-27f45fdf36bf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "20673ee8-572c-4ad1-8a61-27f45fdf36bf" (UID: "20673ee8-572c-4ad1-8a61-27f45fdf36bf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:34:57.403180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.403144    2559 generic.go:356] "Generic (PLEG): container finished" podID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerID="0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412" exitCode=0
Jul 17 05:34:57.403332 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.403222    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" event={"ID":"20673ee8-572c-4ad1-8a61-27f45fdf36bf","Type":"ContainerDied","Data":"0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412"}
Jul 17 05:34:57.403332 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.403322    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" event={"ID":"20673ee8-572c-4ad1-8a61-27f45fdf36bf","Type":"ContainerDied","Data":"040cadc6cc29d36bf516cebf6715272643a4b444dbf50d724f1d54eaed8f08ea"}
Jul 17 05:34:57.403409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.403352    2559 scope.go:117] "RemoveContainer" containerID="0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412"
Jul 17 05:34:57.403442 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.403354    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd"
Jul 17 05:34:57.412933 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.412911    2559 scope.go:117] "RemoveContainer" containerID="0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412"
Jul 17 05:34:57.413261 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:34:57.413238    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412\": container with ID starting with 0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412 not found: ID does not exist" containerID="0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412"
Jul 17 05:34:57.413332 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.413272    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412"} err="failed to get container status \"0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412\": rpc error: code = NotFound desc = could not find container \"0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412\": container with ID starting with 0b1379835a3117f240c712196c5a37526d56d44aefc1c7f82b54faec68d15412 not found: ID does not exist"
Jul 17 05:34:57.425084 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.425049    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69787f6b6f-8vhgd"]
Jul 17 05:34:57.428803 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.428773    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69787f6b6f-8vhgd"]
Jul 17 05:34:57.447708 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447675    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-tls\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447708 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447703    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-registry-certificates\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447708 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447712    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-bound-sa-token\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447963 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447722    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-image-registry-private-configuration\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447963 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447733    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20673ee8-572c-4ad1-8a61-27f45fdf36bf-ca-trust-extracted\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447963 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447741    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20673ee8-572c-4ad1-8a61-27f45fdf36bf-installation-pull-secrets\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447963 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447750    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8kg7\" (UniqueName: \"kubernetes.io/projected/20673ee8-572c-4ad1-8a61-27f45fdf36bf-kube-api-access-j8kg7\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.447963 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.447758    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20673ee8-572c-4ad1-8a61-27f45fdf36bf-trusted-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:34:57.751259 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:34:57.751226    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" path="/var/lib/kubelet/pods/20673ee8-572c-4ad1-8a61-27f45fdf36bf/volumes"
Jul 17 05:35:01.822427 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:01.822396    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:02.167040 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:02.166931    2559 patch_prober.go:28] interesting pod/image-registry-69787f6b6f-8vhgd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 05:35:02.167040 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:02.167015    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69787f6b6f-8vhgd" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 05:35:02.774816 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:02.774777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:02.827373 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:02.827336    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:06.438984 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:06.438949    2559 generic.go:356] "Generic (PLEG): container finished" podID="34afd671-dce9-4fce-8fa0-c265ca199e21" containerID="5ad391cb8083c284ac41b91461faacc186ecc69e09ece3771bd7e482fc9387fc" exitCode=0
Jul 17 05:35:06.439380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:06.439008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-czqrr" event={"ID":"34afd671-dce9-4fce-8fa0-c265ca199e21","Type":"ContainerDied","Data":"5ad391cb8083c284ac41b91461faacc186ecc69e09ece3771bd7e482fc9387fc"}
Jul 17 05:35:06.439380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:06.439340    2559 scope.go:117] "RemoveContainer" containerID="5ad391cb8083c284ac41b91461faacc186ecc69e09ece3771bd7e482fc9387fc"
Jul 17 05:35:07.449273 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:07.449235    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-czqrr" event={"ID":"34afd671-dce9-4fce-8fa0-c265ca199e21","Type":"ContainerStarted","Data":"18d58347b791388c64a87d19cf9ae73788cba7c9f06c4f86709534e362065272"}
Jul 17 05:35:19.667503 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:19.667466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:19.767636 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:19.767596    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:35.283809 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:35.283752    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:35.303448 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:35.303407    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:35.578526 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:35.578439    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:44.831834 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:44.831798    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:44.883461 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:44.883422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:45.165081 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:45.164963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:45.221235 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:45.221200    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:45.311954 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:45.311918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:45.371683 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:45.371650    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:45.911472 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:45.911429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:53.347850 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.347751    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:35:53.349039 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.348963    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="prometheus" containerID="cri-o://7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4" gracePeriod=600
Jul 17 05:35:53.349198 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.349136    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43" gracePeriod=600
Jul 17 05:35:53.349198 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.349191    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy" containerID="cri-o://9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c" gracePeriod=600
Jul 17 05:35:53.349315 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.349270    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-web" containerID="cri-o://73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a" gracePeriod=600
Jul 17 05:35:53.349363 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.349339    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="thanos-sidecar" containerID="cri-o://b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f" gracePeriod=600
Jul 17 05:35:53.349427 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.349407    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="config-reloader" containerID="cri-o://0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a" gracePeriod=600
Jul 17 05:35:53.617099 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.617069    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:53.636409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636361    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43" exitCode=0
Jul 17 05:35:53.636409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636396    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c" exitCode=0
Jul 17 05:35:53.636409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636406    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a" exitCode=0
Jul 17 05:35:53.636409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636416    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f" exitCode=0
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636424    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a" exitCode=0
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636432    2559 generic.go:356] "Generic (PLEG): container finished" podID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4" exitCode=0
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636458    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636483    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"}
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636575    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"}
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"}
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636647    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"}
Jul 17 05:35:53.636720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636703    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"}
Jul 17 05:35:53.637160 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"}
Jul 17 05:35:53.637160 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.636787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9db49aa1-9f71-44c6-b68e-a371832e4b0b","Type":"ContainerDied","Data":"2aeb4a4004195fae21af04b6f7510be881927bbd691e1441b9507a45094f82fc"}
Jul 17 05:35:53.646063 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.646034    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.656912 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.656702    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.669735 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.669695    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.682745 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.682713    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.690669 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.690641    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.701319 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.701294    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.710011 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.709989    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.710336 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.710317    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.710388 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.710347    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"} err="failed to get container status \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist"
Jul 17 05:35:53.710388 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.710367    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.710623 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.710604    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.710668 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.710633    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"} err="failed to get container status \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist"
Jul 17 05:35:53.710668 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.710651    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.710937 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.710916    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.711018 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.710947    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"} err="failed to get container status \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist"
Jul 17 05:35:53.711018 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.710968    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.711232 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.711216    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.711271 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711238    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"} err="failed to get container status \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist"
Jul 17 05:35:53.711271 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711253    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.711458 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.711441    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.711504 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711464    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"} err="failed to get container status \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist"
Jul 17 05:35:53.711504 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711478    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.711665 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.711649    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.711710 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711671    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"} err="failed to get container status \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist"
Jul 17 05:35:53.711710 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711681    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.711905 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711887    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-web-config\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.711905 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:35:53.711895    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-tls\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711918    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"} err="failed to get container status \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist"
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711932    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711942    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-grpc-tls\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.711967    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-serving-certs-ca-bundle\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712029    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-0\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712054    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config-out\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712087    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-kube-rbac-proxy\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712113    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712148    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-trusted-ca-bundle\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712190    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-2\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712223    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-kubelet-serving-ca-bundle\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712252    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-metrics-client-certs\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712302    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-metrics-client-ca\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712328    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-1\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712362    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-db\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.712409 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712391    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-tls-assets\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.713348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712438    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-thanos-prometheus-http-client-file\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.713348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712475    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.713348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4wdn\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-kube-api-access-d4wdn\") pod \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\" (UID: \"9db49aa1-9f71-44c6-b68e-a371832e4b0b\") "
Jul 17 05:35:53.713348 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.712999    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.713560 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.713460    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"} err="failed to get container status \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist"
Jul 17 05:35:53.713560 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.713487    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.713672 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.713560    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.714203 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.713736    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.714203 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.713987    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.714203 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.714138    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.714436 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.714410    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.714436 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.714420    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"} err="failed to get container status \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist"
Jul 17 05:35:53.714555 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.714447    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.714911 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.714883    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"} err="failed to get container status \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist"
Jul 17 05:35:53.715012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.714914    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.715640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.715233    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"} err="failed to get container status \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist"
Jul 17 05:35:53.715640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.715259    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.715640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.715595    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:53.715832 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.715782    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:53.715832 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.715773    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"} err="failed to get container status \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist"
Jul 17 05:35:53.715832 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.715821    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.716137 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716106    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"} err="failed to get container status \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist"
Jul 17 05:35:53.716137 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716124    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-kube-api-access-d4wdn" (OuterVolumeSpecName: "kube-api-access-d4wdn") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "kube-api-access-d4wdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:35:53.716272 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716139    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.716523 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716416    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"} err="failed to get container status \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist"
Jul 17 05:35:53.716523 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716443    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.716523 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716474    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.716770 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716687    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"} err="failed to get container status \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist"
Jul 17 05:35:53.716770 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716716    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.716770 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.716752    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config-out" (OuterVolumeSpecName: "config-out") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:53.717108 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717083    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"} err="failed to get container status \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist"
Jul 17 05:35:53.717186 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717112    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.717519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717378    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"} err="failed to get container status \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist"
Jul 17 05:35:53.717519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717405    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.717519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717482    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.717823 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717672    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"} err="failed to get container status \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist"
Jul 17 05:35:53.717823 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717694    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.718257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.717985    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"} err="failed to get container status \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist"
Jul 17 05:35:53.718257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718008    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.718257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718144    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.718257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718223    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"} err="failed to get container status \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist"
Jul 17 05:35:53.718257 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718246    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.718541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718366    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.718541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718416    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.718541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718420    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:35:53.718541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718517    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"} err="failed to get container status \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist"
Jul 17 05:35:53.718541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718539    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.718783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718609    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.718783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718631    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config" (OuterVolumeSpecName: "config") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.718849 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718789    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"} err="failed to get container status \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist"
Jul 17 05:35:53.718849 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718805    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.719014 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.718992    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.719106 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719085    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"} err="failed to get container status \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist"
Jul 17 05:35:53.719173 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719109    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.719374 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719356    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"} err="failed to get container status \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist"
Jul 17 05:35:53.719436 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719375    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.719588 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719571    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"} err="failed to get container status \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist"
Jul 17 05:35:53.719655 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719589    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.719782 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719765    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"} err="failed to get container status \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist"
Jul 17 05:35:53.719846 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.719783    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.720020 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720001    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"} err="failed to get container status \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist"
Jul 17 05:35:53.720071 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720021    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.720229 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720203    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"} err="failed to get container status \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist"
Jul 17 05:35:53.720229 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720220    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.720447 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720429    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"} err="failed to get container status \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist"
Jul 17 05:35:53.720492 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720448    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.720682 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720663    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"} err="failed to get container status \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist"
Jul 17 05:35:53.720757 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720684    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.721016 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.720996    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"} err="failed to get container status \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist"
Jul 17 05:35:53.721016 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721015    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.721267 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721249    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"} err="failed to get container status \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist"
Jul 17 05:35:53.721314 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721268    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.721491 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721472    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"} err="failed to get container status \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist"
Jul 17 05:35:53.721552 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721492    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.721727 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721711    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"} err="failed to get container status \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist"
Jul 17 05:35:53.721797 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721730    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.721998 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721980    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"} err="failed to get container status \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist"
Jul 17 05:35:53.722056 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.721998    2559 scope.go:117] "RemoveContainer" containerID="eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"
Jul 17 05:35:53.722216 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722194    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43"} err="failed to get container status \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": rpc error: code = NotFound desc = could not find container \"eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43\": container with ID starting with eb42a2f3656dfb28119967999d3c274ebba1ec78e8e7ee41ac761d96ad50ea43 not found: ID does not exist"
Jul 17 05:35:53.722277 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722217    2559 scope.go:117] "RemoveContainer" containerID="9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"
Jul 17 05:35:53.722458 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722439    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c"} err="failed to get container status \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": rpc error: code = NotFound desc = could not find container \"9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c\": container with ID starting with 9a78e07528bc4d433036bcc93c88419215e5beb036eef553832c2daf006c970c not found: ID does not exist"
Jul 17 05:35:53.722507 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722459    2559 scope.go:117] "RemoveContainer" containerID="73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"
Jul 17 05:35:53.722682 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722665    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a"} err="failed to get container status \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": rpc error: code = NotFound desc = could not find container \"73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a\": container with ID starting with 73d7ed6afc91e1885dd3cab9e92206f3647aa01137644ffa1de4c4278108214a not found: ID does not exist"
Jul 17 05:35:53.722750 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722683    2559 scope.go:117] "RemoveContainer" containerID="b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"
Jul 17 05:35:53.722931 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722913    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f"} err="failed to get container status \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": rpc error: code = NotFound desc = could not find container \"b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f\": container with ID starting with b7e02782715087de4cd719e9157d056b84fee1fba63ea07b1ff60e299513a42f not found: ID does not exist"
Jul 17 05:35:53.722983 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.722932    2559 scope.go:117] "RemoveContainer" containerID="0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"
Jul 17 05:35:53.723140 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.723123    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a"} err="failed to get container status \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": rpc error: code = NotFound desc = could not find container \"0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a\": container with ID starting with 0bec13445477ce7192f994d2e1193830006053ff01f2d4cb6bcd423e7440d81a not found: ID does not exist"
Jul 17 05:35:53.723190 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.723141    2559 scope.go:117] "RemoveContainer" containerID="7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"
Jul 17 05:35:53.723378 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.723358    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4"} err="failed to get container status \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": rpc error: code = NotFound desc = could not find container \"7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4\": container with ID starting with 7d8c4873fe4d8aa62376ab4b92d27474456767d08e5637793cb5ed6d74afe4e4 not found: ID does not exist"
Jul 17 05:35:53.723423 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.723379    2559 scope.go:117] "RemoveContainer" containerID="9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"
Jul 17 05:35:53.723582 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.723565    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7"} err="failed to get container status \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": rpc error: code = NotFound desc = could not find container \"9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7\": container with ID starting with 9c84b0af2cf9d3a113ecdde6e3a3c68395d96bc9187de2b3e4f3d0c7b22a1db7 not found: ID does not exist"
Jul 17 05:35:53.727701 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.727673    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-web-config" (OuterVolumeSpecName: "web-config") pod "9db49aa1-9f71-44c6-b68e-a371832e4b0b" (UID: "9db49aa1-9f71-44c6-b68e-a371832e4b0b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:35:53.813250 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813205    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-metrics-client-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813271    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813287    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-db\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813300    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-tls-assets\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813312    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813324    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813337    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4wdn\" (UniqueName: \"kubernetes.io/projected/9db49aa1-9f71-44c6-b68e-a371832e4b0b-kube-api-access-d4wdn\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813349    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-web-config\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813361    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813373    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-grpc-tls\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813385    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813399    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813412    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813424    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9db49aa1-9f71-44c6-b68e-a371832e4b0b-config-out\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813437    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-kube-rbac-proxy\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813936 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813450    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813936 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813464    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813936 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813476    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813936 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813489    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9db49aa1-9f71-44c6-b68e-a371832e4b0b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.813936 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.813501    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9db49aa1-9f71-44c6-b68e-a371832e4b0b-secret-metrics-client-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:35:53.954870 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.954765    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:35:53.961222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.961191    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:35:53.985418 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985380    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:35:53.985665 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985652    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-thanos"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985666    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-thanos"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985674    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-web"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985680    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-web"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985688    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerName="registry"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985693    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerName="registry"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985701    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="config-reloader"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985706    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="config-reloader"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985712    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="thanos-sidecar"
Jul 17 05:35:53.985718 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985717    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="thanos-sidecar"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985725    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="init-config-reloader"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985730    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="init-config-reloader"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985737    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="prometheus"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985742    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="prometheus"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985748    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985753    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985794    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="prometheus"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985801    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="thanos-sidecar"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985809    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-thanos"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985814    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="20673ee8-572c-4ad1-8a61-27f45fdf36bf" containerName="registry"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985820    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy-web"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985825    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="kube-rbac-proxy"
Jul 17 05:35:53.986023 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.985830    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" containerName="config-reloader"
Jul 17 05:35:53.997248 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.997218    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:53.999928 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.999897    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 17 05:35:53.999928 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.999898    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 17 05:35:53.999928 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:53.999931    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 17 05:35:54.000217 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000065    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 17 05:35:54.000217 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000072    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 05:35:54.000320 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000218    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 17 05:35:54.000320 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000277    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 17 05:35:54.000471 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000431    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4ug2f32of0sv9\""
Jul 17 05:35:54.000661 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000481    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 17 05:35:54.000661 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000485    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 17 05:35:54.000661 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000551    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 17 05:35:54.000661 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000560    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 17 05:35:54.000846 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000831    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 17 05:35:54.000994 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.000977    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 17 05:35:54.001092 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.001073    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xqbrz\""
Jul 17 05:35:54.004716 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.004660    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 17 05:35:54.006197 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.006175    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:35:54.007122 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.007096    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 17 05:35:54.015284 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015261    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015390 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015298    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015390 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015329    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-config\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015478    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7b86a9f-605a-4397-b4f5-db298d1200f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/e7b86a9f-605a-4397-b4f5-db298d1200f8-kube-api-access-xbr2g\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015880    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7b86a9f-605a-4397-b4f5-db298d1200f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.015971 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.015898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116387 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116387 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116387 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116424    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116587    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7b86a9f-605a-4397-b4f5-db298d1200f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.116706 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116739    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116769    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/e7b86a9f-605a-4397-b4f5-db298d1200f8-kube-api-access-xbr2g\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116797    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7b86a9f-605a-4397-b4f5-db298d1200f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116901    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.116967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117043    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117528 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-config\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117528 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117528 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117332    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117528 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.117723 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.117589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.119533 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.118664    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.119533 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.118921    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.119974 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.119924    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7b86a9f-605a-4397-b4f5-db298d1200f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.120098 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.120035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.120098 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.120063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.120916 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.120697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7b86a9f-605a-4397-b4f5-db298d1200f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.120916 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.120729    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.120916 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.120742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.121305 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.121276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-config\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.121647 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.121624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.121799 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.121778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e7b86a9f-605a-4397-b4f5-db298d1200f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.122237 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.122215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.122321 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.122286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.122395 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.122372    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.123096 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.123080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e7b86a9f-605a-4397-b4f5-db298d1200f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.125069 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.125049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/e7b86a9f-605a-4397-b4f5-db298d1200f8-kube-api-access-xbr2g\") pod \"prometheus-k8s-0\" (UID: \"e7b86a9f-605a-4397-b4f5-db298d1200f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.309450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.309405    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:35:54.458951 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.458912    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 05:35:54.462903 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:35:54.462873    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b86a9f_605a_4397_b4f5_db298d1200f8.slice/crio-c3ecb3d487f9dfc3c3c3dd0bf8f922f35d7ce6379016bc939b2e613c3389b718 WatchSource:0}: Error finding container c3ecb3d487f9dfc3c3c3dd0bf8f922f35d7ce6379016bc939b2e613c3389b718: Status 404 returned error can't find the container with id c3ecb3d487f9dfc3c3c3dd0bf8f922f35d7ce6379016bc939b2e613c3389b718
Jul 17 05:35:54.642247 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.642149    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7b86a9f-605a-4397-b4f5-db298d1200f8" containerID="d01a92ffa554d6590e83e9a39e4acde2b38d1fcc14be9588fc6e02764193ed0c" exitCode=0
Jul 17 05:35:54.642429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.642243    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerDied","Data":"d01a92ffa554d6590e83e9a39e4acde2b38d1fcc14be9588fc6e02764193ed0c"}
Jul 17 05:35:54.642429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:54.642331    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"c3ecb3d487f9dfc3c3c3dd0bf8f922f35d7ce6379016bc939b2e613c3389b718"}
Jul 17 05:35:55.650729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.650696    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"53f1ef279ae8c1bcfdb2a83bcf6c981d4da9ee76dca57cd574d81afffc29f2c4"}
Jul 17 05:35:55.651167 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.650788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"dc5384c2a6ddfcf74e17cc0c8a779a5c038a57bfd0a343fe092df21a157b6db5"}
Jul 17 05:35:55.651167 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.650815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"e88cb9c7e49d27977b44d997e186ff1e021bf124da4e25767f9180b9022fad83"}
Jul 17 05:35:55.651167 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.650840    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"b6b4fba7983b0ffc11065f5d985caf5eae76b764815252a326a77f9cc733dc9d"}
Jul 17 05:35:55.651167 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.650996    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"96d4f461f436d4fd6807b436909ed2f76de128016cc23eb499071bd5cbaf8cba"}
Jul 17 05:35:55.651167 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.651024    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7b86a9f-605a-4397-b4f5-db298d1200f8","Type":"ContainerStarted","Data":"521a48bba04e236099ad85f13fa1b88a99241dc1fa30fa9e675657dca006409d"}
Jul 17 05:35:55.678172 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.678103    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.658409666 podStartE2EDuration="2.678079197s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.669531ms" isStatefulPod=false podCreationTimestamp="2026-07-17 05:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:35:55.675394457 +0000 UTC m=+180.453877419" watchObservedRunningTime="2026-07-17 05:35:55.678079197 +0000 UTC m=+180.456562158"
Jul 17 05:35:55.750845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:55.750807    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9db49aa1-9f71-44c6-b68e-a371832e4b0b" path="/var/lib/kubelet/pods/9db49aa1-9f71-44c6-b68e-a371832e4b0b/volumes"
Jul 17 05:35:56.064160 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:56.064123    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:56.164504 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:56.164465    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:56.220160 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:56.220124    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:56.741120 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:56.741083    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:56.820211 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:56.820177    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:35:59.310095 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:35:59.310049    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:36:04.671269 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:04.671229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:36:04.771994 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:04.771952    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:36:37.263330 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:37.263291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:36:51.075276 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:51.075236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:36:51.174136 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:51.174095    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:36:54.310323 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:54.310280    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:36:54.327193 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:54.327155    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:36:54.890820 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:54.890792    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 05:36:57.079602 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:36:57.079563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:12.643406 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:12.643372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:13.441375 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:13.441339    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:16.262549 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:16.262510    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:16.320371 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:16.320330    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:18.196994 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:18.196903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:18.295353 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:18.295317    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:18.718923 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:18.718888    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:18.825699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:18.825659    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:20.204072 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:20.204033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:20.609490 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:20.609452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:23.257643 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:23.257603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:23.327769 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:23.327734    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:28.632494 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:28.632455    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:28.685262 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:28.685214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:29.007244 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:29.007203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:29.666214 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:29.666181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:29.765255 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:29.765219    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:31.423567 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.423528    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:31.767242 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.767200    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"]
Jul 17 05:37:31.770386 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.770367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:31.773074 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.773039    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:37:31.774033 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.774011    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-69jrt\""
Jul 17 05:37:31.774157 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.774038    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 05:37:31.781126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.781101    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"]
Jul 17 05:37:31.933160 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.933114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js6tz\" (UniqueName: \"kubernetes.io/projected/ad56b8ef-f9e4-452c-9209-41ba236254c3-kube-api-access-js6tz\") pod \"openshift-lws-operator-fd8ccff4c-rxlfw\" (UID: \"ad56b8ef-f9e4-452c-9209-41ba236254c3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:31.933344 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:31.933173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad56b8ef-f9e4-452c-9209-41ba236254c3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rxlfw\" (UID: \"ad56b8ef-f9e4-452c-9209-41ba236254c3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:32.033905 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:32.033791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad56b8ef-f9e4-452c-9209-41ba236254c3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rxlfw\" (UID: \"ad56b8ef-f9e4-452c-9209-41ba236254c3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:32.033905 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:32.033893    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6tz\" (UniqueName: \"kubernetes.io/projected/ad56b8ef-f9e4-452c-9209-41ba236254c3-kube-api-access-js6tz\") pod \"openshift-lws-operator-fd8ccff4c-rxlfw\" (UID: \"ad56b8ef-f9e4-452c-9209-41ba236254c3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:32.034280 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:32.034260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad56b8ef-f9e4-452c-9209-41ba236254c3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rxlfw\" (UID: \"ad56b8ef-f9e4-452c-9209-41ba236254c3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:32.042147 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:32.042118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js6tz\" (UniqueName: \"kubernetes.io/projected/ad56b8ef-f9e4-452c-9209-41ba236254c3-kube-api-access-js6tz\") pod \"openshift-lws-operator-fd8ccff4c-rxlfw\" (UID: \"ad56b8ef-f9e4-452c-9209-41ba236254c3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:32.092015 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:32.091979    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"
Jul 17 05:37:32.230425 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:32.230394    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw"]
Jul 17 05:37:32.233284 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:37:32.233254    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad56b8ef_f9e4_452c_9209_41ba236254c3.slice/crio-f667bfa82fa0f04861c9ecffd15cc9cfec2125d3967182c12d31d34dbdc0d81d WatchSource:0}: Error finding container f667bfa82fa0f04861c9ecffd15cc9cfec2125d3967182c12d31d34dbdc0d81d: Status 404 returned error can't find the container with id f667bfa82fa0f04861c9ecffd15cc9cfec2125d3967182c12d31d34dbdc0d81d
Jul 17 05:37:33.021625 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:33.021573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw" event={"ID":"ad56b8ef-f9e4-452c-9209-41ba236254c3","Type":"ContainerStarted","Data":"f667bfa82fa0f04861c9ecffd15cc9cfec2125d3967182c12d31d34dbdc0d81d"}
Jul 17 05:37:35.031383 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:35.031340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw" event={"ID":"ad56b8ef-f9e4-452c-9209-41ba236254c3","Type":"ContainerStarted","Data":"a905993efaba6121686e80a92d286cd1d1c0ed9e450a77a5a711716676d04103"}
Jul 17 05:37:35.048139 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:35.048029    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rxlfw" podStartSLOduration=1.494847737 podStartE2EDuration="4.048011486s" totalImagesPullingTime="2.553163749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:37:35.045895472 +0000 UTC m=+279.824378432" watchObservedRunningTime="2026-07-17 05:37:35.048011486 +0000 UTC m=+279.826494446"
Jul 17 05:37:35.811094 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:35.811061    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:35.914470 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:35.914436    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:37.696244 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:37.696206    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:50.073263 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:50.073218    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:50.176808 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:50.176778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:50.690430 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:50.690393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:50.937293 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:50.937260    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.031699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.031638    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.048395 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.048362    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"]
Jul 17 05:37:51.051977 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.051952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.056812 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.056785    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jtkvf\""
Jul 17 05:37:51.057006 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.056810    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 05:37:51.057006 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.056818    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 05:37:51.057006 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.056842    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 05:37:51.057130 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.057031    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 05:37:51.073952 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.073920    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"]
Jul 17 05:37:51.092676 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.092634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4cae1386-805f-4d01-b349-f7fa0f25e25c-manifests\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.092676 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.092680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4cae1386-805f-4d01-b349-f7fa0f25e25c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.092977 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.092724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cae1386-805f-4d01-b349-f7fa0f25e25c-tmp\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.092977 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.092760    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4cae1386-805f-4d01-b349-f7fa0f25e25c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.092977 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.092817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsj28\" (UniqueName: \"kubernetes.io/projected/4cae1386-805f-4d01-b349-f7fa0f25e25c-kube-api-access-vsj28\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.164325 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.164267    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.193245 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.193196    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cae1386-805f-4d01-b349-f7fa0f25e25c-tmp\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.193429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.193340    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4cae1386-805f-4d01-b349-f7fa0f25e25c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.193429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.193378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsj28\" (UniqueName: \"kubernetes.io/projected/4cae1386-805f-4d01-b349-f7fa0f25e25c-kube-api-access-vsj28\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.193429 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.193400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4cae1386-805f-4d01-b349-f7fa0f25e25c-manifests\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.193584 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.193455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4cae1386-805f-4d01-b349-f7fa0f25e25c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.193831 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.193807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4cae1386-805f-4d01-b349-f7fa0f25e25c-manifests\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.195688 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.195663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cae1386-805f-4d01-b349-f7fa0f25e25c-tmp\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.195809 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.195791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4cae1386-805f-4d01-b349-f7fa0f25e25c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.195890 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.195850    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4cae1386-805f-4d01-b349-f7fa0f25e25c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.201910 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.201879    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsj28\" (UniqueName: \"kubernetes.io/projected/4cae1386-805f-4d01-b349-f7fa0f25e25c-kube-api-access-vsj28\") pod \"opendatahub-operator-controller-manager-6d4d7869b5-vqhc9\" (UID: \"4cae1386-805f-4d01-b349-f7fa0f25e25c\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.264229 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.264191    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.363451 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.363356    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:51.369951 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.369917    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.466752 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.466689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.509941 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.509828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"]
Jul 17 05:37:51.513051 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:37:51.513019    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cae1386_805f_4d01_b349_f7fa0f25e25c.slice/crio-42c999999a5055ff464d58fac60829b89c46663f0cd01e759b12723fe68dec94 WatchSource:0}: Error finding container 42c999999a5055ff464d58fac60829b89c46663f0cd01e759b12723fe68dec94: Status 404 returned error can't find the container with id 42c999999a5055ff464d58fac60829b89c46663f0cd01e759b12723fe68dec94
Jul 17 05:37:51.581209 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.581170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.676709 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.676618    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:51.766722 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:51.766689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:37:52.100342 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:52.100287    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9" event={"ID":"4cae1386-805f-4d01-b349-f7fa0f25e25c","Type":"ContainerStarted","Data":"42c999999a5055ff464d58fac60829b89c46663f0cd01e759b12723fe68dec94"}
Jul 17 05:37:55.116127 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:55.116089    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9" event={"ID":"4cae1386-805f-4d01-b349-f7fa0f25e25c","Type":"ContainerStarted","Data":"d79d91f95c711bc9c0aefd4b4cf4f9822ff0459163363946712632f2a0054b66"}
Jul 17 05:37:55.654537 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:55.654502    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 05:37:56.121817 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:56.121778    2559 generic.go:356] "Generic (PLEG): container finished" podID="4cae1386-805f-4d01-b349-f7fa0f25e25c" containerID="d79d91f95c711bc9c0aefd4b4cf4f9822ff0459163363946712632f2a0054b66" exitCode=0
Jul 17 05:37:56.124448 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:56.121881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9" event={"ID":"4cae1386-805f-4d01-b349-f7fa0f25e25c","Type":"ContainerDied","Data":"d79d91f95c711bc9c0aefd4b4cf4f9822ff0459163363946712632f2a0054b66"}
Jul 17 05:37:57.128526 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:57.128491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9" event={"ID":"4cae1386-805f-4d01-b349-f7fa0f25e25c","Type":"ContainerStarted","Data":"d83f18b86fceef091a9dbed1d79ebed7f0478d443208b9c1b737a5852174f279"}
Jul 17 05:37:57.128957 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:57.128637    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:37:57.154335 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:57.154283    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9" podStartSLOduration=1.764838267 podStartE2EDuration="6.154265028s" totalImagesPullingTime="3.214975225s" totalInitContainerRuntime="1.174451536s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:37:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:37:57.15347379 +0000 UTC m=+301.931956747" watchObservedRunningTime="2026-07-17 05:37:57.154265028 +0000 UTC m=+301.932747988"
Jul 17 05:37:59.980038 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:37:59.979990    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:00.081751 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:00.081710    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:08.134515 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:08.134474    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d4d7869b5-vqhc9"
Jul 17 05:38:10.335460 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.335419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:10.709116 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.709030    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"]
Jul 17 05:38:10.829948 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.829908    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"]
Jul 17 05:38:10.830141 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.829963    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:10.832996 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.832969    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mkh8b\""
Jul 17 05:38:10.834066 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.834043    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 17 05:38:10.834221 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.834042    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 17 05:38:10.834221 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.834048    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 17 05:38:10.958113 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.958076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/523a88cd-80d2-4a3c-931a-84e88a0a805c-metrics-cert\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:10.958317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.958141    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8hks\" (UniqueName: \"kubernetes.io/projected/523a88cd-80d2-4a3c-931a-84e88a0a805c-kube-api-access-q8hks\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:10.958317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.958200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/523a88cd-80d2-4a3c-931a-84e88a0a805c-manager-config\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:10.958317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:10.958243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/523a88cd-80d2-4a3c-931a-84e88a0a805c-cert\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.059184 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.059143    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/523a88cd-80d2-4a3c-931a-84e88a0a805c-cert\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.059455 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.059232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/523a88cd-80d2-4a3c-931a-84e88a0a805c-metrics-cert\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.059455 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.059269    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8hks\" (UniqueName: \"kubernetes.io/projected/523a88cd-80d2-4a3c-931a-84e88a0a805c-kube-api-access-q8hks\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.059455 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.059300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/523a88cd-80d2-4a3c-931a-84e88a0a805c-manager-config\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.060000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.059980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/523a88cd-80d2-4a3c-931a-84e88a0a805c-manager-config\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.061715 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.061692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/523a88cd-80d2-4a3c-931a-84e88a0a805c-cert\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.061715 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.061711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/523a88cd-80d2-4a3c-931a-84e88a0a805c-metrics-cert\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.067440 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.067405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8hks\" (UniqueName: \"kubernetes.io/projected/523a88cd-80d2-4a3c-931a-84e88a0a805c-kube-api-access-q8hks\") pod \"lws-controller-manager-6687b4bb84-4dsgv\" (UID: \"523a88cd-80d2-4a3c-931a-84e88a0a805c\") " pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.140387 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.140344    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:11.291030 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.290964    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"]
Jul 17 05:38:11.294263 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:38:11.294226    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523a88cd_80d2_4a3c_931a_84e88a0a805c.slice/crio-9841d6c108e81a90379b9af5bffe31e0c8bd9b786b8de5277b9711612231fd9d WatchSource:0}: Error finding container 9841d6c108e81a90379b9af5bffe31e0c8bd9b786b8de5277b9711612231fd9d: Status 404 returned error can't find the container with id 9841d6c108e81a90379b9af5bffe31e0c8bd9b786b8de5277b9711612231fd9d
Jul 17 05:38:11.297139 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.297118    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:38:11.673545 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.673510    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:11.722875 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.722816    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"]
Jul 17 05:38:11.725585 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.725561    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.728179 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.728138    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 05:38:11.728351 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.728219    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-48wpn\""
Jul 17 05:38:11.728351 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.728217    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 05:38:11.728351 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.728302    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 05:38:11.729233 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.729214    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 05:38:11.739531 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.739498    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"]
Jul 17 05:38:11.765301 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.765252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvbz7\" (UniqueName: \"kubernetes.io/projected/d21763a2-6561-4174-b85e-0a6c36d3f1c8-kube-api-access-dvbz7\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.765506 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.765312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tls-certs\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.765506 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.765392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tmp\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.866532 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.866480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvbz7\" (UniqueName: \"kubernetes.io/projected/d21763a2-6561-4174-b85e-0a6c36d3f1c8-kube-api-access-dvbz7\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.866751 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.866557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tls-certs\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.866751 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.866584    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tmp\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.866751 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:38:11.866737    2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 17 05:38:11.866940 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:38:11.866808    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tls-certs podName:d21763a2-6561-4174-b85e-0a6c36d3f1c8 nodeName:}" failed. No retries permitted until 2026-07-17 05:38:12.366787954 +0000 UTC m=+317.145270892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tls-certs") pod "kube-auth-proxy-86f645db4-kbsm9" (UID: "d21763a2-6561-4174-b85e-0a6c36d3f1c8") : secret "kube-auth-proxy-tls" not found
Jul 17 05:38:11.869027 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.869002    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tmp\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.878674 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.878636    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvbz7\" (UniqueName: \"kubernetes.io/projected/d21763a2-6561-4174-b85e-0a6c36d3f1c8-kube-api-access-dvbz7\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:11.953161 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:11.953068    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:12.194234 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.194194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv" event={"ID":"523a88cd-80d2-4a3c-931a-84e88a0a805c","Type":"ContainerStarted","Data":"9841d6c108e81a90379b9af5bffe31e0c8bd9b786b8de5277b9711612231fd9d"}
Jul 17 05:38:12.371557 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.371504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tls-certs\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:12.374279 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.374236    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21763a2-6561-4174-b85e-0a6c36d3f1c8-tls-certs\") pod \"kube-auth-proxy-86f645db4-kbsm9\" (UID: \"d21763a2-6561-4174-b85e-0a6c36d3f1c8\") " pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:12.567790 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.567742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:12.636768 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.636675    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"
Jul 17 05:38:12.744011 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.743971    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:12.789126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.789084    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86f645db4-kbsm9"]
Jul 17 05:38:12.795190 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:38:12.795152    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21763a2_6561_4174_b85e_0a6c36d3f1c8.slice/crio-db89e5a892bbc88650c9287db9bd8c1fc17672638a814eb91ab1512532962eb5 WatchSource:0}: Error finding container db89e5a892bbc88650c9287db9bd8c1fc17672638a814eb91ab1512532962eb5: Status 404 returned error can't find the container with id db89e5a892bbc88650c9287db9bd8c1fc17672638a814eb91ab1512532962eb5
Jul 17 05:38:12.842903 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:12.842836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:13.200680 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:13.200648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9" event={"ID":"d21763a2-6561-4174-b85e-0a6c36d3f1c8","Type":"ContainerStarted","Data":"db89e5a892bbc88650c9287db9bd8c1fc17672638a814eb91ab1512532962eb5"}
Jul 17 05:38:13.691173 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:13.691139    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:16.222465 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:16.222356    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9" event={"ID":"d21763a2-6561-4174-b85e-0a6c36d3f1c8","Type":"ContainerStarted","Data":"d30cadcb71c45765cf8a37c6df629096b3a3b24319d6bea0a5a25c09f30635a6"}
Jul 17 05:38:16.241803 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:16.241743    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86f645db4-kbsm9" podStartSLOduration=2.074394926 podStartE2EDuration="5.241726573s" totalImagesPullingTime="3.167331647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:38:16.240120413 +0000 UTC m=+321.018603375" watchObservedRunningTime="2026-07-17 05:38:16.241726573 +0000 UTC m=+321.020209581"
Jul 17 05:38:18.397689 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:18.397633    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:18.484809 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:18.484749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:19.046695 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:19.046650    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:19.238659 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:19.238623    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv" event={"ID":"523a88cd-80d2-4a3c-931a-84e88a0a805c","Type":"ContainerStarted","Data":"e043b106bdf739af4389b671c39a4aee7b6d16e0ff04955ede1bbb72652d4168"}
Jul 17 05:38:19.238843 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:19.238780    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:38:19.265986 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:19.265921    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv" podStartSLOduration=1.443235874 podStartE2EDuration="9.265901666s" totalImagesPullingTime="7.822665792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:38:19.263643823 +0000 UTC m=+324.042126787" watchObservedRunningTime="2026-07-17 05:38:19.265901666 +0000 UTC m=+324.044384632"
Jul 17 05:38:19.316365 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:19.316266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:19.409634 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:19.409593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:21.074789 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:21.074749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:21.192345 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:21.192306    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:21.446545 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:21.446453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:21.609993 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:21.609956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:22.391192 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:22.391148    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:22.492703 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:22.492664    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:22.867674 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:22.867636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:24.092565 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:24.092507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:38:30.245137 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:38:30.245102    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6687b4bb84-4dsgv"
Jul 17 05:39:12.028966 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:12.028835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:12.097892 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:12.097839    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:12.248146 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:12.248085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:12.998831 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:12.998778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:13.207479 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:13.207440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:38.702803 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:38.702758    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:38.756178 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:38.756131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:40.739556 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:40.739518    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:41.334367 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:41.334327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:46.012242 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:46.012208    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:46.138707 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:46.138670    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:46.231359 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:46.231318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:48.223356 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:48.223319    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:48.758275 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:48.758239    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:49.771737 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:49.771696    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:51.467254 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:51.467207    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:53.344415 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:53.344377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:55.201893 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:55.201817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:56.353120 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:56.353087    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:56.454222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:56.454172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:56.802039 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:56.802002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:56.886722 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:56.886681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:57.345095 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:57.345043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:57.447530 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:57.447479    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:39:57.504012 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:39:57.503978    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:00.660306 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:00.660262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:00.810815 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:00.810762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:01.272478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.272439    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"]
Jul 17 05:40:01.275984 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.275961    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.278707 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.278677    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 05:40:01.278843 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.278802    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 17 05:40:01.279719 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.279698    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 17 05:40:01.279819 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.279698    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 05:40:01.279819 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.279728    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mmw6m\""
Jul 17 05:40:01.288464 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.288440    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"]
Jul 17 05:40:01.292200 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.292171    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:01.312196 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.312157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ddf00f8-ce20-435e-a016-9ed114c4fb85-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.312410 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.312209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ddf00f8-ce20-435e-a016-9ed114c4fb85-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.312410 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.312310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knsjx\" (UniqueName: \"kubernetes.io/projected/1ddf00f8-ce20-435e-a016-9ed114c4fb85-kube-api-access-knsjx\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.413461 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.413419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ddf00f8-ce20-435e-a016-9ed114c4fb85-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.413687 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.413474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ddf00f8-ce20-435e-a016-9ed114c4fb85-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.413687 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.413507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsjx\" (UniqueName: \"kubernetes.io/projected/1ddf00f8-ce20-435e-a016-9ed114c4fb85-kube-api-access-knsjx\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.413687 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:40:01.413651    2559 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 17 05:40:01.413894 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:40:01.413738    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ddf00f8-ce20-435e-a016-9ed114c4fb85-plugin-serving-cert podName:1ddf00f8-ce20-435e-a016-9ed114c4fb85 nodeName:}" failed. No retries permitted until 2026-07-17 05:40:01.913718003 +0000 UTC m=+426.692200963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/1ddf00f8-ce20-435e-a016-9ed114c4fb85-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-vw58v" (UID: "1ddf00f8-ce20-435e-a016-9ed114c4fb85") : secret "plugin-serving-cert" not found
Jul 17 05:40:01.414240 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.414214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ddf00f8-ce20-435e-a016-9ed114c4fb85-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.423571 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.423537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knsjx\" (UniqueName: \"kubernetes.io/projected/1ddf00f8-ce20-435e-a016-9ed114c4fb85-kube-api-access-knsjx\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.568123 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.567993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:01.918252 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.918153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ddf00f8-ce20-435e-a016-9ed114c4fb85-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:01.920626 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:01.920601    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ddf00f8-ce20-435e-a016-9ed114c4fb85-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vw58v\" (UID: \"1ddf00f8-ce20-435e-a016-9ed114c4fb85\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:02.064519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:02.064485    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:02.185497 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:02.185402    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"
Jul 17 05:40:02.334184 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:02.334144    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v"]
Jul 17 05:40:02.337720 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:40:02.337693    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ddf00f8_ce20_435e_a016_9ed114c4fb85.slice/crio-d18988ca2055900beedbad831ac9558c23897feb7da76b21f3e8ca7b654b1675 WatchSource:0}: Error finding container d18988ca2055900beedbad831ac9558c23897feb7da76b21f3e8ca7b654b1675: Status 404 returned error can't find the container with id d18988ca2055900beedbad831ac9558c23897feb7da76b21f3e8ca7b654b1675
Jul 17 05:40:02.691783 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:02.691750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v" event={"ID":"1ddf00f8-ce20-435e-a016-9ed114c4fb85","Type":"ContainerStarted","Data":"d18988ca2055900beedbad831ac9558c23897feb7da76b21f3e8ca7b654b1675"}
Jul 17 05:40:04.344078 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:04.344033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:04.453658 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:04.453617    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:11.285654 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:11.285592    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:11.465640 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:11.465577    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:12.231915 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.231869    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"]
Jul 17 05:40:12.263601 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.263548    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"]
Jul 17 05:40:12.263794 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.263693    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.266493 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.266462    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kvzrw\""
Jul 17 05:40:12.315621 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.315557    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a953da8-a54e-4775-a407-91bc9a860bde-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hd2hv\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.316127 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.315743    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xflvs\" (UniqueName: \"kubernetes.io/projected/0a953da8-a54e-4775-a407-91bc9a860bde-kube-api-access-xflvs\") pod \"kuadrant-operator-controller-manager-86878c769f-hd2hv\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.416809 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.416761    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a953da8-a54e-4775-a407-91bc9a860bde-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hd2hv\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.417003 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.416842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xflvs\" (UniqueName: \"kubernetes.io/projected/0a953da8-a54e-4775-a407-91bc9a860bde-kube-api-access-xflvs\") pod \"kuadrant-operator-controller-manager-86878c769f-hd2hv\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.417244 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.417217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a953da8-a54e-4775-a407-91bc9a860bde-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hd2hv\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.426514 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.426484    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xflvs\" (UniqueName: \"kubernetes.io/projected/0a953da8-a54e-4775-a407-91bc9a860bde-kube-api-access-xflvs\") pod \"kuadrant-operator-controller-manager-86878c769f-hd2hv\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:12.579617 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:12.579511    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:15.018533 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:15.018499    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:15.112412 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:15.112371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:15.223395 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:15.223355    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:15.317763 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:15.317675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:16.842911 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:16.842876    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:16.944150 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:16.944108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:22.131128 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:22.131085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:22.238031 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:22.237977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:24.388707 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:24.388668    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"]
Jul 17 05:40:24.558226 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:40:24.558140    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a953da8_a54e_4775_a407_91bc9a860bde.slice/crio-aae10b9eab1b3729af00b4c6b1ecf3341e993082d0bbcc47249640d5f42f12bf WatchSource:0}: Error finding container aae10b9eab1b3729af00b4c6b1ecf3341e993082d0bbcc47249640d5f42f12bf: Status 404 returned error can't find the container with id aae10b9eab1b3729af00b4c6b1ecf3341e993082d0bbcc47249640d5f42f12bf
Jul 17 05:40:24.810029 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:24.809927    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v" event={"ID":"1ddf00f8-ce20-435e-a016-9ed114c4fb85","Type":"ContainerStarted","Data":"6ce2ee3a1bdc30b897cc79ae0dc418cb5c62f609f156be2e5ada2b216bb7e441"}
Jul 17 05:40:24.811289 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:24.811258    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv" event={"ID":"0a953da8-a54e-4775-a407-91bc9a860bde","Type":"ContainerStarted","Data":"aae10b9eab1b3729af00b4c6b1ecf3341e993082d0bbcc47249640d5f42f12bf"}
Jul 17 05:40:24.832450 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:24.832397    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vw58v" podStartSLOduration=1.551695898 podStartE2EDuration="23.83237906s" totalImagesPullingTime="22.280683162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:40:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:40:24.830488972 +0000 UTC m=+449.608971933" watchObservedRunningTime="2026-07-17 05:40:24.83237906 +0000 UTC m=+449.610862021"
Jul 17 05:40:28.238579 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:28.238533    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:28.435586 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:28.435552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:29.161199 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:29.161129    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:29.267537 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:29.267495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:32.860020 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:32.859983    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv" event={"ID":"0a953da8-a54e-4775-a407-91bc9a860bde","Type":"ContainerStarted","Data":"383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5"}
Jul 17 05:40:32.860460 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:32.860099    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:32.882215 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:32.882153    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv" podStartSLOduration=13.590837725 podStartE2EDuration="20.88213454s" totalImagesPullingTime="7.291296815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:40:32.879628732 +0000 UTC m=+457.658111692" watchObservedRunningTime="2026-07-17 05:40:32.88213454 +0000 UTC m=+457.660617499"
Jul 17 05:40:32.965845 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:32.965808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:33.029960 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:33.029922    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:33.571188 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:33.571154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:33.629132 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:33.629100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:35.970253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:35.970215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:36.071148 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:36.071102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:41.170716 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:41.170672    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:41.236125 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:41.236086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:43.867093 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:43.867057    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:40:47.737455 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:47.737420    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:40:47.838063 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:40:47.838022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:00.306806 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:00.306768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:00.621301 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:00.621209    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:02.538983 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:02.538942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:02.642059 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:02.642019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:02.741897 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:02.741839    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:03.353699 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.353657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:03.455183 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.455145    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:03.604319 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.604234    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4pksv"]
Jul 17 05:41:03.649806 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.649773    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4pksv"]
Jul 17 05:41:03.650008 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.649896    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:03.652697 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.652658    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rqjkh\""
Jul 17 05:41:03.773459 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.773417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qztp\" (UniqueName: \"kubernetes.io/projected/59070820-1339-49fc-a7b0-c9ccc5faea78-kube-api-access-6qztp\") pod \"authorino-75d556bd77-4pksv\" (UID: \"59070820-1339-49fc-a7b0-c9ccc5faea78\") " pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:03.874153 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.874058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qztp\" (UniqueName: \"kubernetes.io/projected/59070820-1339-49fc-a7b0-c9ccc5faea78-kube-api-access-6qztp\") pod \"authorino-75d556bd77-4pksv\" (UID: \"59070820-1339-49fc-a7b0-c9ccc5faea78\") " pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:03.882249 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.882206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qztp\" (UniqueName: \"kubernetes.io/projected/59070820-1339-49fc-a7b0-c9ccc5faea78-kube-api-access-6qztp\") pod \"authorino-75d556bd77-4pksv\" (UID: \"59070820-1339-49fc-a7b0-c9ccc5faea78\") " pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:03.959997 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:03.959948    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:04.095197 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:04.095018    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4pksv"]
Jul 17 05:41:04.098299 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:41:04.098268    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59070820_1339_49fc_a7b0_c9ccc5faea78.slice/crio-a96d9fda37c64025ca7dfea2adf0543d1de39676ef324ce374129df43b65dcf8 WatchSource:0}: Error finding container a96d9fda37c64025ca7dfea2adf0543d1de39676ef324ce374129df43b65dcf8: Status 404 returned error can't find the container with id a96d9fda37c64025ca7dfea2adf0543d1de39676ef324ce374129df43b65dcf8
Jul 17 05:41:05.004365 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:05.004324    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4pksv" event={"ID":"59070820-1339-49fc-a7b0-c9ccc5faea78","Type":"ContainerStarted","Data":"a96d9fda37c64025ca7dfea2adf0543d1de39676ef324ce374129df43b65dcf8"}
Jul 17 05:41:08.020832 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:08.020793    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4pksv" event={"ID":"59070820-1339-49fc-a7b0-c9ccc5faea78","Type":"ContainerStarted","Data":"1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2"}
Jul 17 05:41:08.053519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:08.053466    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4pksv" podStartSLOduration=2.0655141439999998 podStartE2EDuration="5.053449217s" totalImagesPullingTime="2.987935073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:08.052700683 +0000 UTC m=+492.831183644" watchObservedRunningTime="2026-07-17 05:41:08.053449217 +0000 UTC m=+492.831932176"
Jul 17 05:41:10.328635 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:10.328594    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:10.382319 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:10.382281    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:18.700738 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:18.700691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:18.806411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:18.806367    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:19.234314 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:19.234271    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:30.203496 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:30.203449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:31.894187 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:31.894138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:32.299264 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:32.299212    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:34.623925 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:34.623890    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4pksv"]
Jul 17 05:41:34.624341 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:34.624069    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4pksv" podUID="59070820-1339-49fc-a7b0-c9ccc5faea78" containerName="authorino" containerID="cri-o://1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2" gracePeriod=30
Jul 17 05:41:34.877078 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:34.877048    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:34.956512 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:34.956479    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qztp\" (UniqueName: \"kubernetes.io/projected/59070820-1339-49fc-a7b0-c9ccc5faea78-kube-api-access-6qztp\") pod \"59070820-1339-49fc-a7b0-c9ccc5faea78\" (UID: \"59070820-1339-49fc-a7b0-c9ccc5faea78\") "
Jul 17 05:41:34.958832 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:34.958796    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59070820-1339-49fc-a7b0-c9ccc5faea78-kube-api-access-6qztp" (OuterVolumeSpecName: "kube-api-access-6qztp") pod "59070820-1339-49fc-a7b0-c9ccc5faea78" (UID: "59070820-1339-49fc-a7b0-c9ccc5faea78"). InnerVolumeSpecName "kube-api-access-6qztp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:41:35.058090 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.057991    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qztp\" (UniqueName: \"kubernetes.io/projected/59070820-1339-49fc-a7b0-c9ccc5faea78-kube-api-access-6qztp\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:41:35.138137 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.138096    2559 generic.go:356] "Generic (PLEG): container finished" podID="59070820-1339-49fc-a7b0-c9ccc5faea78" containerID="1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2" exitCode=0
Jul 17 05:41:35.138317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.138140    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4pksv"
Jul 17 05:41:35.138317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.138141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4pksv" event={"ID":"59070820-1339-49fc-a7b0-c9ccc5faea78","Type":"ContainerDied","Data":"1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2"}
Jul 17 05:41:35.138317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.138190    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4pksv" event={"ID":"59070820-1339-49fc-a7b0-c9ccc5faea78","Type":"ContainerDied","Data":"a96d9fda37c64025ca7dfea2adf0543d1de39676ef324ce374129df43b65dcf8"}
Jul 17 05:41:35.138317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.138214    2559 scope.go:117] "RemoveContainer" containerID="1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2"
Jul 17 05:41:35.146813 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.146793    2559 scope.go:117] "RemoveContainer" containerID="1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2"
Jul 17 05:41:35.147143 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:41:35.147117    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2\": container with ID starting with 1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2 not found: ID does not exist" containerID="1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2"
Jul 17 05:41:35.147228 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.147157    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2"} err="failed to get container status \"1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2\": rpc error: code = NotFound desc = could not find container \"1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2\": container with ID starting with 1fb5d0a12d7c3f91cad0fc63a5a0bed89bebc7b9c5632676545261aabbf3ffa2 not found: ID does not exist"
Jul 17 05:41:35.159322 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.159275    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4pksv"]
Jul 17 05:41:35.161132 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.161108    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4pksv"]
Jul 17 05:41:35.751126 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:35.751089    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59070820-1339-49fc-a7b0-c9ccc5faea78" path="/var/lib/kubelet/pods/59070820-1339-49fc-a7b0-c9ccc5faea78/volumes"
Jul 17 05:41:39.491336 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.491299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:39.498682 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.498654    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5578d5449c-gw8nk"]
Jul 17 05:41:39.499081 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.499063    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59070820-1339-49fc-a7b0-c9ccc5faea78" containerName="authorino"
Jul 17 05:41:39.499178 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.499083    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="59070820-1339-49fc-a7b0-c9ccc5faea78" containerName="authorino"
Jul 17 05:41:39.499178 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.499159    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="59070820-1339-49fc-a7b0-c9ccc5faea78" containerName="authorino"
Jul 17 05:41:39.503739 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.503715    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:39.506383 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.506355    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 05:41:39.506532 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.506356    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xcgwf\""
Jul 17 05:41:39.506532 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.506356    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 05:41:39.513627 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.513601    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5578d5449c-gw8nk"]
Jul 17 05:41:39.557966 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.557928    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:39.599326 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.599292    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:39.599509 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.599371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csjfr\" (UniqueName: \"kubernetes.io/projected/52174bdc-f553-472d-94c9-b04424b77fc2-kube-api-access-csjfr\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:39.626150 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.626114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:39.700406 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.700371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:39.700576 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.700462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csjfr\" (UniqueName: \"kubernetes.io/projected/52174bdc-f553-472d-94c9-b04424b77fc2-kube-api-access-csjfr\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:39.700576 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:41:39.700525    2559 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 17 05:41:39.700647 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:41:39.700613    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert podName:52174bdc-f553-472d-94c9-b04424b77fc2 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:40.200595597 +0000 UTC m=+524.979078555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert") pod "maas-controller-5578d5449c-gw8nk" (UID: "52174bdc-f553-472d-94c9-b04424b77fc2") : secret "maas-controller-webhook-cert" not found
Jul 17 05:41:39.711274 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:39.711241    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csjfr\" (UniqueName: \"kubernetes.io/projected/52174bdc-f553-472d-94c9-b04424b77fc2-kube-api-access-csjfr\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:40.111814 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:40.111774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:40.205937 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:40.205901    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:40.208610 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:40.208583    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert\") pod \"maas-controller-5578d5449c-gw8nk\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") " pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:40.415717 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:40.415623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:40.559810 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:40.559778    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5578d5449c-gw8nk"]
Jul 17 05:41:40.562684 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:41:40.562649    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52174bdc_f553_472d_94c9_b04424b77fc2.slice/crio-e0b75effde7bd303b240ea47d2f28fea08b0837d09407dfea30341b33e2e5701 WatchSource:0}: Error finding container e0b75effde7bd303b240ea47d2f28fea08b0837d09407dfea30341b33e2e5701: Status 404 returned error can't find the container with id e0b75effde7bd303b240ea47d2f28fea08b0837d09407dfea30341b33e2e5701
Jul 17 05:41:41.167801 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:41.167754    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5578d5449c-gw8nk" event={"ID":"52174bdc-f553-472d-94c9-b04424b77fc2","Type":"ContainerStarted","Data":"e0b75effde7bd303b240ea47d2f28fea08b0837d09407dfea30341b33e2e5701"}
Jul 17 05:41:43.259013 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:43.258963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:43.362274 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:43.362186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:41:44.186062 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:44.186028    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5578d5449c-gw8nk" event={"ID":"52174bdc-f553-472d-94c9-b04424b77fc2","Type":"ContainerStarted","Data":"06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c"}
Jul 17 05:41:44.186253 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:44.186100    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:44.204197 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:44.204130    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5578d5449c-gw8nk" podStartSLOduration=2.655272222 podStartE2EDuration="5.20411064s" totalImagesPullingTime="2.548838418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:44.202043791 +0000 UTC m=+528.980526762" watchObservedRunningTime="2026-07-17 05:41:44.20411064 +0000 UTC m=+528.982593603"
Jul 17 05:41:54.693563 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:54.693473    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5578d5449c-gw8nk"]
Jul 17 05:41:54.694014 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:54.693674    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5578d5449c-gw8nk" podUID="52174bdc-f553-472d-94c9-b04424b77fc2" containerName="manager" containerID="cri-o://06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c" gracePeriod=30
Jul 17 05:41:54.695503 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:54.695476    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:54.947194 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:54.947115    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:55.044195 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.044151    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert\") pod \"52174bdc-f553-472d-94c9-b04424b77fc2\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") "
Jul 17 05:41:55.044195 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.044195    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csjfr\" (UniqueName: \"kubernetes.io/projected/52174bdc-f553-472d-94c9-b04424b77fc2-kube-api-access-csjfr\") pod \"52174bdc-f553-472d-94c9-b04424b77fc2\" (UID: \"52174bdc-f553-472d-94c9-b04424b77fc2\") "
Jul 17 05:41:55.046443 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.046408    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52174bdc-f553-472d-94c9-b04424b77fc2-kube-api-access-csjfr" (OuterVolumeSpecName: "kube-api-access-csjfr") pod "52174bdc-f553-472d-94c9-b04424b77fc2" (UID: "52174bdc-f553-472d-94c9-b04424b77fc2"). InnerVolumeSpecName "kube-api-access-csjfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:41:55.046568 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.046471    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "52174bdc-f553-472d-94c9-b04424b77fc2" (UID: "52174bdc-f553-472d-94c9-b04424b77fc2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:41:55.145275 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.145232    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52174bdc-f553-472d-94c9-b04424b77fc2-webhook-cert\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:41:55.145275 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.145267    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csjfr\" (UniqueName: \"kubernetes.io/projected/52174bdc-f553-472d-94c9-b04424b77fc2-kube-api-access-csjfr\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:41:55.235549 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.235514    2559 generic.go:356] "Generic (PLEG): container finished" podID="52174bdc-f553-472d-94c9-b04424b77fc2" containerID="06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c" exitCode=0
Jul 17 05:41:55.235724 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.235566    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5578d5449c-gw8nk"
Jul 17 05:41:55.235724 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.235592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5578d5449c-gw8nk" event={"ID":"52174bdc-f553-472d-94c9-b04424b77fc2","Type":"ContainerDied","Data":"06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c"}
Jul 17 05:41:55.235724 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.235662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5578d5449c-gw8nk" event={"ID":"52174bdc-f553-472d-94c9-b04424b77fc2","Type":"ContainerDied","Data":"e0b75effde7bd303b240ea47d2f28fea08b0837d09407dfea30341b33e2e5701"}
Jul 17 05:41:55.235724 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.235684    2559 scope.go:117] "RemoveContainer" containerID="06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c"
Jul 17 05:41:55.244714 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.244692    2559 scope.go:117] "RemoveContainer" containerID="06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c"
Jul 17 05:41:55.245021 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:41:55.245002    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c\": container with ID starting with 06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c not found: ID does not exist" containerID="06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c"
Jul 17 05:41:55.245092 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.245030    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c"} err="failed to get container status \"06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c\": rpc error: code = NotFound desc = could not find container \"06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c\": container with ID starting with 06436307560a87b5f543681bf8c0dbf5c71611296636d91df0a54d51d558fc0c not found: ID does not exist"
Jul 17 05:41:55.257042 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.257014    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5578d5449c-gw8nk"]
Jul 17 05:41:55.260434 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.260409    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5578d5449c-gw8nk"]
Jul 17 05:41:55.751317 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:41:55.751286    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52174bdc-f553-472d-94c9-b04424b77fc2" path="/var/lib/kubelet/pods/52174bdc-f553-472d-94c9-b04424b77fc2/volumes"
Jul 17 05:42:04.749683 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:04.749637    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:14.948421 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:14.948373    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:15.046192 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:15.046154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:15.436982 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:15.436943    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:15.624018 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:15.623978    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:15.832016 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:15.831980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:15.904218 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:15.904177    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:26.910245 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:26.910204    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:27.000274 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:27.000245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:35.149622 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:35.149580    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:51.115832 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.115795    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"]
Jul 17 05:42:51.116338 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.116132    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52174bdc-f553-472d-94c9-b04424b77fc2" containerName="manager"
Jul 17 05:42:51.116338 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.116143    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="52174bdc-f553-472d-94c9-b04424b77fc2" containerName="manager"
Jul 17 05:42:51.116338 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.116235    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="52174bdc-f553-472d-94c9-b04424b77fc2" containerName="manager"
Jul 17 05:42:51.119157 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.119140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.122722 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.122444    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vzdzz\""
Jul 17 05:42:51.122722 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.122628    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 05:42:51.124432 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.123743    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 05:42:51.124432 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.124010    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 05:42:51.131581 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.131557    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"]
Jul 17 05:42:51.220496 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.220454    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:42:51.236240 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.236200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f05f67-8bcb-4cb3-a920-2587f24d324e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.236240 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.236246    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kph6\" (UniqueName: \"kubernetes.io/projected/40f05f67-8bcb-4cb3-a920-2587f24d324e-kube-api-access-5kph6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.236438 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.236313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.236438 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.236363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.236438 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.236397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.236541 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.236443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.337478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.337434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.337478 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.337484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.337729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.337540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f05f67-8bcb-4cb3-a920-2587f24d324e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.337729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.337567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kph6\" (UniqueName: \"kubernetes.io/projected/40f05f67-8bcb-4cb3-a920-2587f24d324e-kube-api-access-5kph6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.337729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.337606    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.337729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.337648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.338041 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.338016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.338118 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.338029    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.338118 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.338100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.339737 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.339715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f05f67-8bcb-4cb3-a920-2587f24d324e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.340255 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.340232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f05f67-8bcb-4cb3-a920-2587f24d324e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.345236 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.345213    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kph6\" (UniqueName: \"kubernetes.io/projected/40f05f67-8bcb-4cb3-a920-2587f24d324e-kube-api-access-5kph6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qkq6h\" (UID: \"40f05f67-8bcb-4cb3-a920-2587f24d324e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.433210 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.433109    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:42:51.574729 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:51.574702    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"]
Jul 17 05:42:51.577265 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:42:51.577234    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f05f67_8bcb_4cb3_a920_2587f24d324e.slice/crio-61fbb34a1ecca19b9ddce797b5ca858d889cd5eec1f94fa0b601f4c47e89554c WatchSource:0}: Error finding container 61fbb34a1ecca19b9ddce797b5ca858d889cd5eec1f94fa0b601f4c47e89554c: Status 404 returned error can't find the container with id 61fbb34a1ecca19b9ddce797b5ca858d889cd5eec1f94fa0b601f4c47e89554c
Jul 17 05:42:52.484326 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:52.484279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h" event={"ID":"40f05f67-8bcb-4cb3-a920-2587f24d324e","Type":"ContainerStarted","Data":"61fbb34a1ecca19b9ddce797b5ca858d889cd5eec1f94fa0b601f4c47e89554c"}
Jul 17 05:42:53.490161 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:53.490124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h" event={"ID":"40f05f67-8bcb-4cb3-a920-2587f24d324e","Type":"ContainerStarted","Data":"3fa38e8430db7da5ae5cf2b061fce140fd48bf16ce562c50e0f201af0da9fc92"}
Jul 17 05:42:53.507994 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:42:53.507939    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h" podStartSLOduration=1.513181888 podStartE2EDuration="2.507918891s" totalImagesPullingTime="994.737003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:42:53.506507398 +0000 UTC m=+598.284990359" watchObservedRunningTime="2026-07-17 05:42:53.507918891 +0000 UTC m=+598.286401851"
Jul 17 05:43:01.433358 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:01.433320    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:43:01.435871 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:01.433387    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:43:01.445977 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:01.445947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:43:01.537376 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:01.537347    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qkq6h"
Jul 17 05:43:10.815502 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.815464    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"]
Jul 17 05:43:10.822442 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.822414    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:10.825168 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.825133    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 17 05:43:10.830231 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.830024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"]
Jul 17 05:43:10.847569 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.847529    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:43:10.907782 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.907740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:10.908019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.907807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:10.908019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.907842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:10.908019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.907892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:10.908019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.907959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzlxt\" (UniqueName: \"kubernetes.io/projected/2931593b-a567-42fa-a7b9-69f679231d8f-kube-api-access-wzlxt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:10.908019 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:10.907995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2931593b-a567-42fa-a7b9-69f679231d8f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008386 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008573 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008427    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008573 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzlxt\" (UniqueName: \"kubernetes.io/projected/2931593b-a567-42fa-a7b9-69f679231d8f-kube-api-access-wzlxt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008573 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008516    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2931593b-a567-42fa-a7b9-69f679231d8f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008573 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008762 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008895 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.008987 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.008906    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.009098 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.009076    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.010875 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.010843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2931593b-a567-42fa-a7b9-69f679231d8f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.011075 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.011060    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2931593b-a567-42fa-a7b9-69f679231d8f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.016000 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.015976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzlxt\" (UniqueName: \"kubernetes.io/projected/2931593b-a567-42fa-a7b9-69f679231d8f-kube-api-access-wzlxt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzk6j\" (UID: \"2931593b-a567-42fa-a7b9-69f679231d8f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.134115 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.134021    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:11.272749 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.272720    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"]
Jul 17 05:43:11.275574 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:43:11.275544    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2931593b_a567_42fa_a7b9_69f679231d8f.slice/crio-454d41988d48d45a4324b19d39bc1d49238c2640dc488587f8bada032ae80970 WatchSource:0}: Error finding container 454d41988d48d45a4324b19d39bc1d49238c2640dc488587f8bada032ae80970: Status 404 returned error can't find the container with id 454d41988d48d45a4324b19d39bc1d49238c2640dc488587f8bada032ae80970
Jul 17 05:43:11.571916 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.571879    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j" event={"ID":"2931593b-a567-42fa-a7b9-69f679231d8f","Type":"ContainerStarted","Data":"f8144b6c021da71a8a14e73ee3d76c44fc29a8d0fd2c00d5e6974250ddd799d8"}
Jul 17 05:43:11.572119 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.571945    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j" event={"ID":"2931593b-a567-42fa-a7b9-69f679231d8f","Type":"ContainerStarted","Data":"454d41988d48d45a4324b19d39bc1d49238c2640dc488587f8bada032ae80970"}
Jul 17 05:43:11.588617 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:11.588559    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j" podStartSLOduration=1.414005316 podStartE2EDuration="1.588540145s" totalImagesPullingTime="174.534829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:43:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:43:11.587998844 +0000 UTC m=+616.366481805" watchObservedRunningTime="2026-07-17 05:43:11.588540145 +0000 UTC m=+616.367023104"
Jul 17 05:43:15.721665 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:15.721604    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:43:21.134549 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.134440    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:21.134986 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.134564    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:21.147309 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.147281    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:21.516580 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.516544    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"]
Jul 17 05:43:21.520007 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.519986    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.522341 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.522317    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 17 05:43:21.528667 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.528637    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"]
Jul 17 05:43:21.539486 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.539453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:43:21.601591 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.601558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.601793 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.601610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.601793 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.601696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.601793 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.601724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.601793 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.601761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.601982 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.601798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55d9b\" (UniqueName: \"kubernetes.io/projected/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-kube-api-access-55d9b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.628916 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.628885    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzk6j"
Jul 17 05:43:21.702825 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.702790    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.702851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.702925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.702950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.702995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703049 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.703034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55d9b\" (UniqueName: \"kubernetes.io/projected/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-kube-api-access-55d9b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.703358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703411 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.703387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.703607 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.703457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.705668 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.705642    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.705958 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.705942    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.710544 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.710522    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55d9b\" (UniqueName: \"kubernetes.io/projected/fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2-kube-api-access-55d9b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j5mvz\" (UID: \"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.831758 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.831662    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:21.972720 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.972691    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"]
Jul 17 05:43:21.975269 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:43:21.975228    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2f5051_ea0a_4f4f_98ed_a63cf959a4e2.slice/crio-b560d1583a498f76137d1d85478b03f55c05bb021ad09cda0118b26c3264d50c WatchSource:0}: Error finding container b560d1583a498f76137d1d85478b03f55c05bb021ad09cda0118b26c3264d50c: Status 404 returned error can't find the container with id b560d1583a498f76137d1d85478b03f55c05bb021ad09cda0118b26c3264d50c
Jul 17 05:43:21.977180 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:21.977159    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:43:22.626822 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:22.626756    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz" event={"ID":"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2","Type":"ContainerStarted","Data":"99f907bd1f9a0da682e276381d683ec117e95bc04e119cb8b9b3179dcb646876"}
Jul 17 05:43:22.627325 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:22.626843    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz" event={"ID":"fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2","Type":"ContainerStarted","Data":"b560d1583a498f76137d1d85478b03f55c05bb021ad09cda0118b26c3264d50c"}
Jul 17 05:43:22.646210 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:22.646154    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz" podStartSLOduration=1.456040368 podStartE2EDuration="1.646135487s" totalImagesPullingTime="190.095119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:43:22.644808973 +0000 UTC m=+627.423291933" watchObservedRunningTime="2026-07-17 05:43:22.646135487 +0000 UTC m=+627.424618447"
Jul 17 05:43:30.238311 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:30.238273    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:43:31.832775 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:31.832718    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:31.833255 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:31.832804    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:31.845645 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:31.845613    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:32.688128 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:32.688098    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j5mvz"
Jul 17 05:43:33.409708 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:43:33.409676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:44:11.445232 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:44:11.445185    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:44:39.158254 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:44:39.158217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:44:39.265420 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:44:39.265387    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:44:40.380760 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:44:40.380688    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:44:40.480514 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:44:40.480479    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:45:03.153380 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:45:03.153289    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:45:14.338611 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:45:14.338575    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:45:37.587413 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:45:37.587370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:45:37.641675 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:45:37.641629    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:46:37.755404 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:37.755366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:46:37.856830 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:37.856790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:46:44.680668 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:44.680628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:46:45.916598 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:45.916552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:46:46.038473 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.038436    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7994cc7dcc-qp6wn"]
Jul 17 05:46:46.041742 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.041722    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.044632 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.044604    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xcgwf\""
Jul 17 05:46:46.045584 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.045568    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 05:46:46.045657 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.045613    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 05:46:46.051757 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.051731    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7994cc7dcc-qp6wn"]
Jul 17 05:46:46.142831 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.142789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m22dc\" (UniqueName: \"kubernetes.io/projected/3b4f78b5-d0c5-4378-a649-fbae92327340-kube-api-access-m22dc\") pod \"maas-controller-7994cc7dcc-qp6wn\" (UID: \"3b4f78b5-d0c5-4378-a649-fbae92327340\") " pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.143048 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.142965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b4f78b5-d0c5-4378-a649-fbae92327340-webhook-cert\") pod \"maas-controller-7994cc7dcc-qp6wn\" (UID: \"3b4f78b5-d0c5-4378-a649-fbae92327340\") " pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.244318 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.244281    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m22dc\" (UniqueName: \"kubernetes.io/projected/3b4f78b5-d0c5-4378-a649-fbae92327340-kube-api-access-m22dc\") pod \"maas-controller-7994cc7dcc-qp6wn\" (UID: \"3b4f78b5-d0c5-4378-a649-fbae92327340\") " pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.244506 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.244387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b4f78b5-d0c5-4378-a649-fbae92327340-webhook-cert\") pod \"maas-controller-7994cc7dcc-qp6wn\" (UID: \"3b4f78b5-d0c5-4378-a649-fbae92327340\") " pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.246869 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.246832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b4f78b5-d0c5-4378-a649-fbae92327340-webhook-cert\") pod \"maas-controller-7994cc7dcc-qp6wn\" (UID: \"3b4f78b5-d0c5-4378-a649-fbae92327340\") " pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.254133 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.254108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m22dc\" (UniqueName: \"kubernetes.io/projected/3b4f78b5-d0c5-4378-a649-fbae92327340-kube-api-access-m22dc\") pod \"maas-controller-7994cc7dcc-qp6wn\" (UID: \"3b4f78b5-d0c5-4378-a649-fbae92327340\") " pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.353127 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.353085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:46.497188 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.497162    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7994cc7dcc-qp6wn"]
Jul 17 05:46:46.499382 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:46:46.499354    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4f78b5_d0c5_4378_a649_fbae92327340.slice/crio-27ad9171198f666c1f7766c06523f6d4091a2378fec91da39656e24a7cbe4ed1 WatchSource:0}: Error finding container 27ad9171198f666c1f7766c06523f6d4091a2378fec91da39656e24a7cbe4ed1: Status 404 returned error can't find the container with id 27ad9171198f666c1f7766c06523f6d4091a2378fec91da39656e24a7cbe4ed1
Jul 17 05:46:46.577255 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:46.577221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn" event={"ID":"3b4f78b5-d0c5-4378-a649-fbae92327340","Type":"ContainerStarted","Data":"27ad9171198f666c1f7766c06523f6d4091a2378fec91da39656e24a7cbe4ed1"}
Jul 17 05:46:47.583082 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:47.583038    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn" event={"ID":"3b4f78b5-d0c5-4378-a649-fbae92327340","Type":"ContainerStarted","Data":"9b6d348796b1dea369c4cdbd62da3b4d54fa2c3adfa6a409e939f7051e2b0abc"}
Jul 17 05:46:47.583567 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:47.583139    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:46:47.600973 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:47.600915    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn" podStartSLOduration=1.145676297 podStartE2EDuration="1.600897906s" totalImagesPullingTime="455.221609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:47.598848265 +0000 UTC m=+832.377331224" watchObservedRunningTime="2026-07-17 05:46:47.600897906 +0000 UTC m=+832.379380866"
Jul 17 05:46:58.590256 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:46:58.590223    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7994cc7dcc-qp6wn"
Jul 17 05:47:20.145847 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:20.145807    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:20.245281 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:20.245248    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:20.391728 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:20.391675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:26.141147 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:26.141110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:26.204454 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:26.204415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:26.297679 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:26.297637    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:26.397638 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:26.397525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:26.517747 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:26.517708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:32.137202 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:32.137170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:47:32.195149 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:47:32.195109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:48:49.749952 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:48:49.749762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:49:21.414626 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:49:21.414518    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:51:23.690513 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:23.690462    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"]
Jul 17 05:51:23.691150 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:23.690730    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv" podUID="0a953da8-a54e-4775-a407-91bc9a860bde" containerName="manager" containerID="cri-o://383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5" gracePeriod=10
Jul 17 05:51:27.647157 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.647127    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:51:27.742308 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.742268    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a953da8-a54e-4775-a407-91bc9a860bde-extensions-socket-volume\") pod \"0a953da8-a54e-4775-a407-91bc9a860bde\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") "
Jul 17 05:51:27.742519 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.742345    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xflvs\" (UniqueName: \"kubernetes.io/projected/0a953da8-a54e-4775-a407-91bc9a860bde-kube-api-access-xflvs\") pod \"0a953da8-a54e-4775-a407-91bc9a860bde\" (UID: \"0a953da8-a54e-4775-a407-91bc9a860bde\") "
Jul 17 05:51:27.742750 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.742722    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a953da8-a54e-4775-a407-91bc9a860bde-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a953da8-a54e-4775-a407-91bc9a860bde" (UID: "0a953da8-a54e-4775-a407-91bc9a860bde"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:51:27.744518 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.744488    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a953da8-a54e-4775-a407-91bc9a860bde-kube-api-access-xflvs" (OuterVolumeSpecName: "kube-api-access-xflvs") pod "0a953da8-a54e-4775-a407-91bc9a860bde" (UID: "0a953da8-a54e-4775-a407-91bc9a860bde"). InnerVolumeSpecName "kube-api-access-xflvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:51:27.843191 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.843152    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a953da8-a54e-4775-a407-91bc9a860bde-extensions-socket-volume\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:51:27.843191 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.843186    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xflvs\" (UniqueName: \"kubernetes.io/projected/0a953da8-a54e-4775-a407-91bc9a860bde-kube-api-access-xflvs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 05:51:27.882886 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.882835    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a953da8-a54e-4775-a407-91bc9a860bde" containerID="383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5" exitCode=0
Jul 17 05:51:27.882886 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.882894    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"
Jul 17 05:51:27.883129 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.882928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv" event={"ID":"0a953da8-a54e-4775-a407-91bc9a860bde","Type":"ContainerDied","Data":"383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5"}
Jul 17 05:51:27.883129 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.882985    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv" event={"ID":"0a953da8-a54e-4775-a407-91bc9a860bde","Type":"ContainerDied","Data":"aae10b9eab1b3729af00b4c6b1ecf3341e993082d0bbcc47249640d5f42f12bf"}
Jul 17 05:51:27.883129 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.883014    2559 scope.go:117] "RemoveContainer" containerID="383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5"
Jul 17 05:51:27.892437 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.892417    2559 scope.go:117] "RemoveContainer" containerID="383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5"
Jul 17 05:51:27.892748 ip-10-0-128-189 kubenswrapper[2559]: E0717 05:51:27.892728    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5\": container with ID starting with 383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5 not found: ID does not exist" containerID="383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5"
Jul 17 05:51:27.892823 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.892756    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5"} err="failed to get container status \"383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5\": rpc error: code = NotFound desc = could not find container \"383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5\": container with ID starting with 383b31827315032cda5722cc68ee58728a1257c32a4cfbd7293125ac2ed403d5 not found: ID does not exist"
Jul 17 05:51:27.899239 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.899215    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"]
Jul 17 05:51:27.903242 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:27.903219    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hd2hv"]
Jul 17 05:51:29.751750 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:51:29.751708    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a953da8-a54e-4775-a407-91bc9a860bde" path="/var/lib/kubelet/pods/0a953da8-a54e-4775-a407-91bc9a860bde/volumes"
Jul 17 05:52:24.892476 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.892446    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"]
Jul 17 05:52:24.892914 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.892773    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a953da8-a54e-4775-a407-91bc9a860bde" containerName="manager"
Jul 17 05:52:24.892914 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.892783    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a953da8-a54e-4775-a407-91bc9a860bde" containerName="manager"
Jul 17 05:52:24.892914 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.892849    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a953da8-a54e-4775-a407-91bc9a860bde" containerName="manager"
Jul 17 05:52:24.895762 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.895744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:24.898414 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.898393    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kvzrw\""
Jul 17 05:52:24.912476 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.912443    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"]
Jul 17 05:52:24.960077 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.960031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzb85\" (UniqueName: \"kubernetes.io/projected/9623e809-a106-495e-99d9-818388e7c314-kube-api-access-mzb85\") pod \"kuadrant-operator-controller-manager-86878c769f-hwwlt\" (UID: \"9623e809-a106-495e-99d9-818388e7c314\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:24.960255 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:24.960152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9623e809-a106-495e-99d9-818388e7c314-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwwlt\" (UID: \"9623e809-a106-495e-99d9-818388e7c314\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:25.060785 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:25.060745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9623e809-a106-495e-99d9-818388e7c314-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwwlt\" (UID: \"9623e809-a106-495e-99d9-818388e7c314\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:25.061003 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:25.060825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzb85\" (UniqueName: \"kubernetes.io/projected/9623e809-a106-495e-99d9-818388e7c314-kube-api-access-mzb85\") pod \"kuadrant-operator-controller-manager-86878c769f-hwwlt\" (UID: \"9623e809-a106-495e-99d9-818388e7c314\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:25.061188 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:25.061166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9623e809-a106-495e-99d9-818388e7c314-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwwlt\" (UID: \"9623e809-a106-495e-99d9-818388e7c314\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:25.070128 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:25.070095    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzb85\" (UniqueName: \"kubernetes.io/projected/9623e809-a106-495e-99d9-818388e7c314-kube-api-access-mzb85\") pod \"kuadrant-operator-controller-manager-86878c769f-hwwlt\" (UID: \"9623e809-a106-495e-99d9-818388e7c314\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:25.206015 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:25.205910    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:25.348633 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:25.348596    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"]
Jul 17 05:52:25.352143 ip-10-0-128-189 kubenswrapper[2559]: W0717 05:52:25.352105    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9623e809_a106_495e_99d9_818388e7c314.slice/crio-0ba06c3d4c1e0f85e7264e74b586b0ac008caadb97eaf83486ccf04cc03959d1 WatchSource:0}: Error finding container 0ba06c3d4c1e0f85e7264e74b586b0ac008caadb97eaf83486ccf04cc03959d1: Status 404 returned error can't find the container with id 0ba06c3d4c1e0f85e7264e74b586b0ac008caadb97eaf83486ccf04cc03959d1
Jul 17 05:52:26.147711 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:26.147667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt" event={"ID":"9623e809-a106-495e-99d9-818388e7c314","Type":"ContainerStarted","Data":"01fb3798c30da55f591171339c65856990a71cadf18ddc10a96cda81ce84c54c"}
Jul 17 05:52:26.148265 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:26.147732    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:26.148265 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:26.147749    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt" event={"ID":"9623e809-a106-495e-99d9-818388e7c314","Type":"ContainerStarted","Data":"0ba06c3d4c1e0f85e7264e74b586b0ac008caadb97eaf83486ccf04cc03959d1"}
Jul 17 05:52:26.168653 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:26.168598    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt" podStartSLOduration=2.168577902 podStartE2EDuration="2.168577902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:26.166375929 +0000 UTC m=+1170.944858889" watchObservedRunningTime="2026-07-17 05:52:26.168577902 +0000 UTC m=+1170.947060861"
Jul 17 05:52:37.155533 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:37.155502    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwwlt"
Jul 17 05:52:54.152933 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:54.152896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:52:54.240724 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:52:54.240681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:55:56.407591 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:55:56.407535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:55:57.558254 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:55:57.558217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:56:10.364830 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:56:10.364790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:21.802009 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:21.801961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:21.895382 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:21.895331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:26.680626 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:26.680590    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:26.789439 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:26.789403    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:26.882958 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:26.882918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:27.959513 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:27.959470    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:28.072276 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:28.072241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:28.182072 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:28.182028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:28.297354 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:28.297316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:28.803589 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:28.803553    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:28.904781 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:28.904743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:30.315222 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:30.315179    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 05:58:33.370794 ip-10-0-128-189 kubenswrapper[2559]: I0717 05:58:33.370749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:33.275763 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:33.275729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:35.030989 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:35.030953    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:38.988097 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:38.988047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:39.693965 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:39.693927    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:45.276472 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:45.276423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:45.378631 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:45.378594    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:45.564547 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:45.564430    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:45.691338 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:45.691306    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:45.826323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:45.826243    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:47.247671 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:47.247636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:47.349951 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:47.349916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:47.730248 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:47.730210    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:47.830198 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:47.830168    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:50.055939 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:50.055901    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:50.153034 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:50.152995    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:01:53.628752 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:01:53.628713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:00.061046 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:00.061011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:07.821716 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:07.821669    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:07.927096 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:07.927063    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:12.947518 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:12.947488    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:13.049951 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:13.049915    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:51.750380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:51.750287    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:51.861133 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:51.861102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:57.660329 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:57.660294    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:02:57.759678 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:02:57.759649    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:03.173668 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:03.173630    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:03.480332 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:03.480294    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:03.543660 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:03.543623    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:03.848049 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:03.848009    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:03.930134 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:03.930093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:04.956442 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:04.956407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:05.041913 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:05.041884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:05.137254 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:05.137216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:05.402237 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:05.402209    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:05.631212 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:05.631184    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:05.734980 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:05.734948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:03:07.303823 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:03:07.303790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:04:42.800186 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:04:42.800151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:04:42.866095 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:04:42.866053    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:05:50.069059 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:05:50.068972    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:07:08.584245 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:07:08.584203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:07:10.131465 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:07:10.131432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:07:10.968698 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:07:10.968661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:07:11.069425 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:07:11.069392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:10.355436 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:10.355352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:10.450328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:10.450291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:15.571260 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:15.571225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:16.438768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:16.438733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:21.861756 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:21.861728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:21.920119 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:21.920082    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:22.082245 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:22.082210    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:23.219574 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:23.219514    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:23.306775 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:23.306734    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:23.379623 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:23.379587    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:23.819250 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:23.819217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:24.148971 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:24.148840    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:24.252128 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:24.252083    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:31.901270 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:31.901239    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:31.957136 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:31.957104    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:09:35.888588 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:09:35.888556    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:09.356199 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:09.356120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:09.443192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:09.443152    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:27.827424 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:27.827382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:28.899236 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:28.899199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:33.784822 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:33.784782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:33.885579 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:33.885532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:39.354233 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:39.354196    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:39.405613 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:39.405583    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:39.896029 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:39.895999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:39.995469 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:39.995435    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:40.096322 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:40.096291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:41.749885 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:41.749817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:47.473687 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:47.473641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:47.583386 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:47.583350    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:47.680114 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:47.680083    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:47.780186 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:47.780086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:48.224302 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:48.224258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:12:54.297740 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:12:54.297709    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:14.290957 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:14.290913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:19.948400 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:19.948301    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:20.009714 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:20.009665    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:20.128043 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:20.127999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:20.228483 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:20.228422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:27.617026 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:27.616982    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:36.083264 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:36.083229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:42.022950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:42.022913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:42.158294 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:42.158263    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:42.328192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:42.328090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:52.825395 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:52.825361    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:53.636779 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:53.636744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:59.458516 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:59.458478    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:59.564369 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:59.564330    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:59.660352 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:59.660319    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:13:59.766319 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:13:59.766286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:00.567512 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:00.567419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:00.667480 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:00.667444    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:00.767036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:00.766996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:02.408396 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:02.408367    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:08.224494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:08.224460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:08.351069 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:08.351033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:08.944053 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:08.944014    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:14.679000 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:14.678961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:14.859754 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:14.859724    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:14.959755 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:14.959667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:15.280712 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:15.280678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:15.474358 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:15.474322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:18.542020 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:18.541983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:18.678016 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:18.677975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:18.798202 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:18.798115    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:18.894494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:18.894461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:19.380089 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:19.380047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:19.480099 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:19.480060    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:23.015601 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:23.015562    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:23.711267 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:23.711233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:23.810183 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:23.810137    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:24.166442 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:24.166404    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:29.946740 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:29.946695    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:30.048057 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:30.048015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:35.001670 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:35.001632    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:40.897081 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:40.897035    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:45.414793 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:45.414756    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:51.407031 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:51.406946    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:51.468301 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:51.468264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:56.105276 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:56.105239    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:56.338484 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:56.338438    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:56.506929 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:56.506892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:56.595155 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:56.595121    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:14:56.693568 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:14:56.693536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:02.624731 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:02.624697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:02.730670 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:02.730631    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:02.841881 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:02.841829    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:03.010933 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:03.010901    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:08.797036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:08.797005    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:08.894145 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:08.894116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:15.717846 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:15.717809    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:16.038689 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:16.038601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:16.137961 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:16.137930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:18.767297 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:18.767257    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:18.842289 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:18.842252    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:18.916787 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:18.916756    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:19.326391 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:19.326357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:19.426367 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:19.426329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:19.527080 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:19.527044    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:23.311100 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:23.311063    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:23.493286 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:23.493255    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:23.548423 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:23.548393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:28.408075 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.408025    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"]
Jul 17 06:15:28.411694 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.411666    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.415397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.415361    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-603a20da\"/\"default-dockercfg-9nqqt\""
Jul 17 06:15:28.415397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.415361    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-603a20da\"/\"openshift-service-ca.crt\""
Jul 17 06:15:28.415603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.415401    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-603a20da\"/\"kube-root-ca.crt\""
Jul 17 06:15:28.415603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.415364    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-603a20da\"/\"e2e-lifecycle-model-603a20da-kserve-self-signed-certs\""
Jul 17 06:15:28.419591 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.419382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"]
Jul 17 06:15:28.439902 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.439853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:15:28.450019 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.449956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-tmp-dir\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.450207 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.450038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz97w\" (UniqueName: \"kubernetes.io/projected/cdf55885-8190-431d-ab52-8f6fa2140021-kube-api-access-pz97w\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.450207 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.450078    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-home\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.450207 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.450155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-dshm\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.450207 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.450201    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-model-cache\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.450389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.450240    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdf55885-8190-431d-ab52-8f6fa2140021-tls-certs\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551349 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-tmp-dir\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551349 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz97w\" (UniqueName: \"kubernetes.io/projected/cdf55885-8190-431d-ab52-8f6fa2140021-kube-api-access-pz97w\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551640 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-home\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551640 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-dshm\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551640 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-model-cache\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551640 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdf55885-8190-431d-ab52-8f6fa2140021-tls-certs\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551881 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551824    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-tmp-dir\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551881 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551873    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-home\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.551966 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.551891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-model-cache\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.553839 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.553813    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-dshm\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.554174 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.554156    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdf55885-8190-431d-ab52-8f6fa2140021-tls-certs\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.559280 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.559259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz97w\" (UniqueName: \"kubernetes.io/projected/cdf55885-8190-431d-ab52-8f6fa2140021-kube-api-access-pz97w\") pod \"e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") " pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.722648 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.722612    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:28.861664 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:28.861504    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"]
Jul 17 06:15:28.864679 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:15:28.864643    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdf55885_8190_431d_ab52_8f6fa2140021.slice/crio-b35791d0b439de00cf13b8eadc3a4d2d6631bde9c7a2fd8aae152cae3e566e21 WatchSource:0}: Error finding container b35791d0b439de00cf13b8eadc3a4d2d6631bde9c7a2fd8aae152cae3e566e21: Status 404 returned error can't find the container with id b35791d0b439de00cf13b8eadc3a4d2d6631bde9c7a2fd8aae152cae3e566e21
Jul 17 06:15:29.499036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:29.498996    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2" event={"ID":"cdf55885-8190-431d-ab52-8f6fa2140021","Type":"ContainerStarted","Data":"b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d"}
Jul 17 06:15:29.499036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:29.499042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2" event={"ID":"cdf55885-8190-431d-ab52-8f6fa2140021","Type":"ContainerStarted","Data":"b35791d0b439de00cf13b8eadc3a4d2d6631bde9c7a2fd8aae152cae3e566e21"}
Jul 17 06:15:29.516372 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:29.516314    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2" podStartSLOduration=1.516294556 podStartE2EDuration="1.516294556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:15:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:15:29.515068401 +0000 UTC m=+2554.293551363" watchObservedRunningTime="2026-07-17 06:15:29.516294556 +0000 UTC m=+2554.294777516"
Jul 17 06:15:38.723374 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:38.723323    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:38.723374 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:38.723385    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:38.736422 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:38.736401    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:15:39.557378 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:15:39.557344    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:16:01.926108 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:01.926065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:16:06.522933 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:06.522900    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:16:11.480268 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:11.480220    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"]
Jul 17 06:16:11.577176 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:11.577119    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:16:26.501136 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.501069    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2" podUID="cdf55885-8190-431d-ab52-8f6fa2140021" containerName="main" containerID="cri-o://b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d" gracePeriod=45
Jul 17 06:16:26.755290 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.755209    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:16:26.770420 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.770388    2559 generic.go:356] "Generic (PLEG): container finished" podID="cdf55885-8190-431d-ab52-8f6fa2140021" containerID="b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d" exitCode=0
Jul 17 06:16:26.770603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.770427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2" event={"ID":"cdf55885-8190-431d-ab52-8f6fa2140021","Type":"ContainerDied","Data":"b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d"}
Jul 17 06:16:26.770603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.770434    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"
Jul 17 06:16:26.770603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.770458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2" event={"ID":"cdf55885-8190-431d-ab52-8f6fa2140021","Type":"ContainerDied","Data":"b35791d0b439de00cf13b8eadc3a4d2d6631bde9c7a2fd8aae152cae3e566e21"}
Jul 17 06:16:26.770603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.770475    2559 scope.go:117] "RemoveContainer" containerID="b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d"
Jul 17 06:16:26.780562 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.780539    2559 scope.go:117] "RemoveContainer" containerID="b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d"
Jul 17 06:16:26.780965 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:16:26.780944    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d\": container with ID starting with b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d not found: ID does not exist" containerID="b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d"
Jul 17 06:16:26.781032 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.780978    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d"} err="failed to get container status \"b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d\": rpc error: code = NotFound desc = could not find container \"b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d\": container with ID starting with b251da6666ddd865bb006a9a72495267370e5a758c9fe46eafa3965dc499b14d not found: ID does not exist"
Jul 17 06:16:26.900848 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.900804    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-dshm\") pod \"cdf55885-8190-431d-ab52-8f6fa2140021\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") "
Jul 17 06:16:26.901084 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.900905    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-home\") pod \"cdf55885-8190-431d-ab52-8f6fa2140021\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") "
Jul 17 06:16:26.901084 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.900944    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz97w\" (UniqueName: \"kubernetes.io/projected/cdf55885-8190-431d-ab52-8f6fa2140021-kube-api-access-pz97w\") pod \"cdf55885-8190-431d-ab52-8f6fa2140021\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") "
Jul 17 06:16:26.901084 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.900969    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdf55885-8190-431d-ab52-8f6fa2140021-tls-certs\") pod \"cdf55885-8190-431d-ab52-8f6fa2140021\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") "
Jul 17 06:16:26.901275 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901099    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-model-cache\") pod \"cdf55885-8190-431d-ab52-8f6fa2140021\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") "
Jul 17 06:16:26.901275 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901154    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-tmp-dir\") pod \"cdf55885-8190-431d-ab52-8f6fa2140021\" (UID: \"cdf55885-8190-431d-ab52-8f6fa2140021\") "
Jul 17 06:16:26.901275 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901163    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-home" (OuterVolumeSpecName: "home") pod "cdf55885-8190-431d-ab52-8f6fa2140021" (UID: "cdf55885-8190-431d-ab52-8f6fa2140021"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:26.901446 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901354    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdf55885-8190-431d-ab52-8f6fa2140021" (UID: "cdf55885-8190-431d-ab52-8f6fa2140021"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:26.901524 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901491    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdf55885-8190-431d-ab52-8f6fa2140021" (UID: "cdf55885-8190-431d-ab52-8f6fa2140021"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:26.901589 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901556    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:16:26.901589 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.901578    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:16:26.903307 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.903283    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf55885-8190-431d-ab52-8f6fa2140021-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdf55885-8190-431d-ab52-8f6fa2140021" (UID: "cdf55885-8190-431d-ab52-8f6fa2140021"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:16:26.903307 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.903300    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-dshm" (OuterVolumeSpecName: "dshm") pod "cdf55885-8190-431d-ab52-8f6fa2140021" (UID: "cdf55885-8190-431d-ab52-8f6fa2140021"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:26.903307 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:26.903305    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf55885-8190-431d-ab52-8f6fa2140021-kube-api-access-pz97w" (OuterVolumeSpecName: "kube-api-access-pz97w") pod "cdf55885-8190-431d-ab52-8f6fa2140021" (UID: "cdf55885-8190-431d-ab52-8f6fa2140021"). InnerVolumeSpecName "kube-api-access-pz97w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:16:27.002950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.002911    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:16:27.002950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.002942    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz97w\" (UniqueName: \"kubernetes.io/projected/cdf55885-8190-431d-ab52-8f6fa2140021-kube-api-access-pz97w\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:16:27.002950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.002953    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdf55885-8190-431d-ab52-8f6fa2140021-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:16:27.002950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.002962    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf55885-8190-431d-ab52-8f6fa2140021-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:16:27.092780 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.092741    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"]
Jul 17 06:16:27.096247 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.096216    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-603a20da/e2e-lifecycle-model-603a20da-kserve-66b4cdcc4c-s7gg2"]
Jul 17 06:16:27.751622 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:27.751589    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf55885-8190-431d-ab52-8f6fa2140021" path="/var/lib/kubelet/pods/cdf55885-8190-431d-ab52-8f6fa2140021/volumes"
Jul 17 06:16:43.756933 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:43.756884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:16:48.401820 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:48.401770    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:16:48.561499 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:48.561469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:16:54.063016 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:16:54.062983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:00.148769 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:00.148732    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:00.249924 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:00.249889    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:08.949565 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:08.949527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:09.181572 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:09.181537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:10.085812 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:10.085781    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:10.185610 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:10.185563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:10.284998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:10.284966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:10.386086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:10.385988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:10.484237 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:10.484206    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:11.260266 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:11.260195    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:11.361099 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:11.361064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:11.457231 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:11.457198    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:12.902883 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:12.902810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:19.164190 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:19.164158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:19.324019 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:19.323979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:19.896363 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:19.896324    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:26.113719 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:26.113683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:26.176393 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:26.176350    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:26.311727 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:26.311685    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:26.396771 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:26.396682    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:36.947583 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:36.947545    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:38.298147 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:38.298100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:17:58.414555 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:17:58.414448    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:05.630431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:05.630398    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:05.727015 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:05.726977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:06.029399 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:06.029364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:06.426034 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:06.425940    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:06.828233 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:06.828164    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:09.987073 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:09.987031    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:10.252527 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:10.252441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:10.600337 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:10.600262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:11.381403 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:11.381364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:11.569218 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:11.569182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:14.474151 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:14.474091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:14.887391 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:14.887305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:15.455753 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:15.455722    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:15.857022 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:15.856981    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:18.953977 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:18.953929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:19.123161 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:19.123127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:19.276565 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:19.276531    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:19.456047 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:19.456012    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:20.220283 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:20.220252    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:20.369348 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:20.369314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:23.478536 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:23.478503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:33.499623 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:33.499586    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:33.599548 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:33.599519    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:38.846208 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.846168    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"]
Jul 17 06:18:38.846610 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.846524    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf55885-8190-431d-ab52-8f6fa2140021" containerName="main"
Jul 17 06:18:38.846610 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.846534    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf55885-8190-431d-ab52-8f6fa2140021" containerName="main"
Jul 17 06:18:38.846610 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.846595    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf55885-8190-431d-ab52-8f6fa2140021" containerName="main"
Jul 17 06:18:38.849496 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.849475    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.854148 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.854118    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c0d19a\"/\"kube-root-ca.crt\""
Jul 17 06:18:38.854148 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.854135    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c0d19a\"/\"openshift-service-ca.crt\""
Jul 17 06:18:38.855560 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.855533    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c0d19a\"/\"test-model-c0d19a-kserve-self-signed-certs\""
Jul 17 06:18:38.857321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.857303    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c0d19a\"/\"default-dockercfg-pglcd\""
Jul 17 06:18:38.864183 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.864155    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"]
Jul 17 06:18:38.877603 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.877579    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:38.897165 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.897132    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-dshm\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.897165 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.897166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-model-cache\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.897398 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.897192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434bdebd-f342-45c4-80da-a8a418e87854-tls-certs\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.897398 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.897271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-tmp-dir\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.897398 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.897346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2xfv\" (UniqueName: \"kubernetes.io/projected/434bdebd-f342-45c4-80da-a8a418e87854-kube-api-access-z2xfv\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.897398 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.897395    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-home\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998203 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-tmp-dir\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998409 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2xfv\" (UniqueName: \"kubernetes.io/projected/434bdebd-f342-45c4-80da-a8a418e87854-kube-api-access-z2xfv\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998409 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998265    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-home\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998409 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998301    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-dshm\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998409 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-model-cache\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998409 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434bdebd-f342-45c4-80da-a8a418e87854-tls-certs\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998747 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-tmp-dir\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998904 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-home\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:38.998904 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:38.998824    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-model-cache\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:39.000736 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:39.000713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-dshm\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:39.000917 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:39.000899    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434bdebd-f342-45c4-80da-a8a418e87854-tls-certs\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:39.008175 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:39.008148    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2xfv\" (UniqueName: \"kubernetes.io/projected/434bdebd-f342-45c4-80da-a8a418e87854-kube-api-access-z2xfv\") pod \"test-model-c0d19a-kserve-b8955bcfc-twz54\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") " pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:39.158804 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:39.158699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:39.307427 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:39.307397    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"]
Jul 17 06:18:39.310100 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:18:39.310061    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod434bdebd_f342_45c4_80da_a8a418e87854.slice/crio-3c0d2a2753fbd397cb1b7bdf0f37bf23902077c7d6f6d9662dfe67ffca2a8b98 WatchSource:0}: Error finding container 3c0d2a2753fbd397cb1b7bdf0f37bf23902077c7d6f6d9662dfe67ffca2a8b98: Status 404 returned error can't find the container with id 3c0d2a2753fbd397cb1b7bdf0f37bf23902077c7d6f6d9662dfe67ffca2a8b98
Jul 17 06:18:39.375148 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:39.375109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54" event={"ID":"434bdebd-f342-45c4-80da-a8a418e87854","Type":"ContainerStarted","Data":"3c0d2a2753fbd397cb1b7bdf0f37bf23902077c7d6f6d9662dfe67ffca2a8b98"}
Jul 17 06:18:40.381049 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:40.381010    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54" event={"ID":"434bdebd-f342-45c4-80da-a8a418e87854","Type":"ContainerStarted","Data":"ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7"}
Jul 17 06:18:40.403684 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:40.403627    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54" podStartSLOduration=2.403608484 podStartE2EDuration="2.403608484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:18:40.401566615 +0000 UTC m=+2745.180049575" watchObservedRunningTime="2026-07-17 06:18:40.403608484 +0000 UTC m=+2745.182091443"
Jul 17 06:18:42.306088 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:42.306019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:42.413323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:42.413286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:18:49.159527 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:49.159484    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:49.159527 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:49.159541    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:49.172474 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:49.172444    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:18:49.434475 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:18:49.434384    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:19:04.131908 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.131845    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"]
Jul 17 06:19:04.135561 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.135542    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.138831 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.138805    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3483d7\"/\"default-dockercfg-bnwdq\""
Jul 17 06:19:04.138974 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.138815    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3483d7\"/\"kube-root-ca.crt\""
Jul 17 06:19:04.139986 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.139969    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3483d7\"/\"openshift-service-ca.crt\""
Jul 17 06:19:04.140074 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.140058    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3483d7\"/\"test-model-3483d7-kserve-self-signed-certs\""
Jul 17 06:19:04.146990 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.146965    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"]
Jul 17 06:19:04.166221 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.166188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:04.233361 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.233318    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbrsg\" (UniqueName: \"kubernetes.io/projected/7d3572cb-7f1b-4939-92b0-ee55d730c490-kube-api-access-mbrsg\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.233578 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.233369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-dshm\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.233578 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.233447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3572cb-7f1b-4939-92b0-ee55d730c490-tls-certs\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.233578 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.233506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-tmp-dir\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.233578 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.233547    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-home\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.233578 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.233571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-model-cache\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334043 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbrsg\" (UniqueName: \"kubernetes.io/projected/7d3572cb-7f1b-4939-92b0-ee55d730c490-kube-api-access-mbrsg\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334043 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-dshm\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334091    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3572cb-7f1b-4939-92b0-ee55d730c490-tls-certs\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-tmp-dir\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334163    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-home\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334284    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-model-cache\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334617 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334563    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-home\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334617 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334601    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-tmp-dir\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.334723 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.334669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-model-cache\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.336555 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.336537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-dshm\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.336842 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.336821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3572cb-7f1b-4939-92b0-ee55d730c490-tls-certs\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.344631 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.344601    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbrsg\" (UniqueName: \"kubernetes.io/projected/7d3572cb-7f1b-4939-92b0-ee55d730c490-kube-api-access-mbrsg\") pod \"test-model-3483d7-kserve-865497fc79-jrhfh\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") " pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.444916 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.444803    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:04.637364 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:04.637329    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"]
Jul 17 06:19:04.640936 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:19:04.640905    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d3572cb_7f1b_4939_92b0_ee55d730c490.slice/crio-5e168d7738e4acc025767a96f555faba8b47a231e29109522dac427ae48b75db WatchSource:0}: Error finding container 5e168d7738e4acc025767a96f555faba8b47a231e29109522dac427ae48b75db: Status 404 returned error can't find the container with id 5e168d7738e4acc025767a96f555faba8b47a231e29109522dac427ae48b75db
Jul 17 06:19:05.498541 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:05.498499    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh" event={"ID":"7d3572cb-7f1b-4939-92b0-ee55d730c490","Type":"ContainerStarted","Data":"aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd"}
Jul 17 06:19:05.498541 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:05.498542    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh" event={"ID":"7d3572cb-7f1b-4939-92b0-ee55d730c490","Type":"ContainerStarted","Data":"5e168d7738e4acc025767a96f555faba8b47a231e29109522dac427ae48b75db"}
Jul 17 06:19:05.519061 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:05.519006    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh" podStartSLOduration=1.518986841 podStartE2EDuration="1.518986841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:19:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:19:05.517984183 +0000 UTC m=+2770.296467142" watchObservedRunningTime="2026-07-17 06:19:05.518986841 +0000 UTC m=+2770.297469801"
Jul 17 06:19:14.446023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:14.445968    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:14.446542 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:14.446038    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:14.458700 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:14.458669    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:14.552850 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:14.552823    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:32.654646 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:32.654605    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"]
Jul 17 06:19:32.700333 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:32.700300    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:33.695662 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:33.695619    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"]
Jul 17 06:19:33.769912 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:33.769880    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:34.598947 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:34.598908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:34.709592 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:34.709555    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:44.383497 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:44.383460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:47.674935 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.674894    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54" podUID="434bdebd-f342-45c4-80da-a8a418e87854" containerName="main" containerID="cri-o://ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7" gracePeriod=45
Jul 17 06:19:47.926545 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.926474    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:19:47.947562 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947523    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2xfv\" (UniqueName: \"kubernetes.io/projected/434bdebd-f342-45c4-80da-a8a418e87854-kube-api-access-z2xfv\") pod \"434bdebd-f342-45c4-80da-a8a418e87854\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") "
Jul 17 06:19:47.947562 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947560    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434bdebd-f342-45c4-80da-a8a418e87854-tls-certs\") pod \"434bdebd-f342-45c4-80da-a8a418e87854\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") "
Jul 17 06:19:47.947832 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947592    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-dshm\") pod \"434bdebd-f342-45c4-80da-a8a418e87854\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") "
Jul 17 06:19:47.947832 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947635    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-tmp-dir\") pod \"434bdebd-f342-45c4-80da-a8a418e87854\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") "
Jul 17 06:19:47.947832 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947687    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-model-cache\") pod \"434bdebd-f342-45c4-80da-a8a418e87854\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") "
Jul 17 06:19:47.947832 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947707    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-home\") pod \"434bdebd-f342-45c4-80da-a8a418e87854\" (UID: \"434bdebd-f342-45c4-80da-a8a418e87854\") "
Jul 17 06:19:47.948105 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947944    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "434bdebd-f342-45c4-80da-a8a418e87854" (UID: "434bdebd-f342-45c4-80da-a8a418e87854"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:47.948105 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.947962    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-model-cache" (OuterVolumeSpecName: "model-cache") pod "434bdebd-f342-45c4-80da-a8a418e87854" (UID: "434bdebd-f342-45c4-80da-a8a418e87854"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:47.948308 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.948230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-home" (OuterVolumeSpecName: "home") pod "434bdebd-f342-45c4-80da-a8a418e87854" (UID: "434bdebd-f342-45c4-80da-a8a418e87854"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:47.950063 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.950040    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/434bdebd-f342-45c4-80da-a8a418e87854-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "434bdebd-f342-45c4-80da-a8a418e87854" (UID: "434bdebd-f342-45c4-80da-a8a418e87854"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:19:47.950187 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.950162    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-dshm" (OuterVolumeSpecName: "dshm") pod "434bdebd-f342-45c4-80da-a8a418e87854" (UID: "434bdebd-f342-45c4-80da-a8a418e87854"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:47.950272 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:47.950226    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434bdebd-f342-45c4-80da-a8a418e87854-kube-api-access-z2xfv" (OuterVolumeSpecName: "kube-api-access-z2xfv") pod "434bdebd-f342-45c4-80da-a8a418e87854" (UID: "434bdebd-f342-45c4-80da-a8a418e87854"). InnerVolumeSpecName "kube-api-access-z2xfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:19:48.048845 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.048807    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:48.048845 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.048840    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:48.048845 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.048849    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2xfv\" (UniqueName: \"kubernetes.io/projected/434bdebd-f342-45c4-80da-a8a418e87854-kube-api-access-z2xfv\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:48.049138 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.048888    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434bdebd-f342-45c4-80da-a8a418e87854-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:48.049138 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.048903    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:48.049138 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.048913    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434bdebd-f342-45c4-80da-a8a418e87854-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:48.710192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.710152    2559 generic.go:356] "Generic (PLEG): container finished" podID="434bdebd-f342-45c4-80da-a8a418e87854" containerID="ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7" exitCode=0
Jul 17 06:19:48.710695 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.710206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54" event={"ID":"434bdebd-f342-45c4-80da-a8a418e87854","Type":"ContainerDied","Data":"ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7"}
Jul 17 06:19:48.710695 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.710217    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"
Jul 17 06:19:48.710695 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.710236    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54" event={"ID":"434bdebd-f342-45c4-80da-a8a418e87854","Type":"ContainerDied","Data":"3c0d2a2753fbd397cb1b7bdf0f37bf23902077c7d6f6d9662dfe67ffca2a8b98"}
Jul 17 06:19:48.710695 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.710252    2559 scope.go:117] "RemoveContainer" containerID="ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7"
Jul 17 06:19:48.716872 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.716804    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh" podUID="7d3572cb-7f1b-4939-92b0-ee55d730c490" containerName="main" containerID="cri-o://aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd" gracePeriod=45
Jul 17 06:19:48.722904 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.722850    2559 scope.go:117] "RemoveContainer" containerID="ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7"
Jul 17 06:19:48.723267 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:19:48.723235    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7\": container with ID starting with ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7 not found: ID does not exist" containerID="ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7"
Jul 17 06:19:48.723355 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.723282    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7"} err="failed to get container status \"ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7\": rpc error: code = NotFound desc = could not find container \"ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7\": container with ID starting with ec294dcdff950c1f95e6ea681ee038a3a9cd5cd4f426bdb4a995bd2c97c6b9a7 not found: ID does not exist"
Jul 17 06:19:48.738746 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.738712    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"]
Jul 17 06:19:48.742602 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.742574    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c0d19a/test-model-c0d19a-kserve-b8955bcfc-twz54"]
Jul 17 06:19:48.961046 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.960977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:48.981440 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:48.981411    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:49.057649 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.057611    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-home\") pod \"7d3572cb-7f1b-4939-92b0-ee55d730c490\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") "
Jul 17 06:19:49.057649 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.057650    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-model-cache\") pod \"7d3572cb-7f1b-4939-92b0-ee55d730c490\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") "
Jul 17 06:19:49.057960 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.057692    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbrsg\" (UniqueName: \"kubernetes.io/projected/7d3572cb-7f1b-4939-92b0-ee55d730c490-kube-api-access-mbrsg\") pod \"7d3572cb-7f1b-4939-92b0-ee55d730c490\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") "
Jul 17 06:19:49.057960 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.057709    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-tmp-dir\") pod \"7d3572cb-7f1b-4939-92b0-ee55d730c490\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") "
Jul 17 06:19:49.057960 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.057737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-dshm\") pod \"7d3572cb-7f1b-4939-92b0-ee55d730c490\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") "
Jul 17 06:19:49.057960 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.057800    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3572cb-7f1b-4939-92b0-ee55d730c490-tls-certs\") pod \"7d3572cb-7f1b-4939-92b0-ee55d730c490\" (UID: \"7d3572cb-7f1b-4939-92b0-ee55d730c490\") "
Jul 17 06:19:49.058162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.058012    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-home" (OuterVolumeSpecName: "home") pod "7d3572cb-7f1b-4939-92b0-ee55d730c490" (UID: "7d3572cb-7f1b-4939-92b0-ee55d730c490"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:49.058162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.058103    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7d3572cb-7f1b-4939-92b0-ee55d730c490" (UID: "7d3572cb-7f1b-4939-92b0-ee55d730c490"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:49.058162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.058119    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-model-cache" (OuterVolumeSpecName: "model-cache") pod "7d3572cb-7f1b-4939-92b0-ee55d730c490" (UID: "7d3572cb-7f1b-4939-92b0-ee55d730c490"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:49.060202 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.060178    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-dshm" (OuterVolumeSpecName: "dshm") pod "7d3572cb-7f1b-4939-92b0-ee55d730c490" (UID: "7d3572cb-7f1b-4939-92b0-ee55d730c490"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:19:49.060311 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.060241    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3572cb-7f1b-4939-92b0-ee55d730c490-kube-api-access-mbrsg" (OuterVolumeSpecName: "kube-api-access-mbrsg") pod "7d3572cb-7f1b-4939-92b0-ee55d730c490" (UID: "7d3572cb-7f1b-4939-92b0-ee55d730c490"). InnerVolumeSpecName "kube-api-access-mbrsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:19:49.060311 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.060256    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3572cb-7f1b-4939-92b0-ee55d730c490-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7d3572cb-7f1b-4939-92b0-ee55d730c490" (UID: "7d3572cb-7f1b-4939-92b0-ee55d730c490"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:19:49.159326 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.159283    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbrsg\" (UniqueName: \"kubernetes.io/projected/7d3572cb-7f1b-4939-92b0-ee55d730c490-kube-api-access-mbrsg\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:49.159494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.159342    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:49.159494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.159354    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:49.159494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.159362    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3572cb-7f1b-4939-92b0-ee55d730c490-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:49.159494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.159370    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:49.159494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.159378    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3572cb-7f1b-4939-92b0-ee55d730c490-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:19:49.716672 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.716638    2559 generic.go:356] "Generic (PLEG): container finished" podID="7d3572cb-7f1b-4939-92b0-ee55d730c490" containerID="aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd" exitCode=0
Jul 17 06:19:49.716672 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.716678    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"
Jul 17 06:19:49.717231 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.716716    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh" event={"ID":"7d3572cb-7f1b-4939-92b0-ee55d730c490","Type":"ContainerDied","Data":"aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd"}
Jul 17 06:19:49.717231 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.716761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh" event={"ID":"7d3572cb-7f1b-4939-92b0-ee55d730c490","Type":"ContainerDied","Data":"5e168d7738e4acc025767a96f555faba8b47a231e29109522dac427ae48b75db"}
Jul 17 06:19:49.717231 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.716778    2559 scope.go:117] "RemoveContainer" containerID="aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd"
Jul 17 06:19:49.725708 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.725685    2559 scope.go:117] "RemoveContainer" containerID="aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd"
Jul 17 06:19:49.726043 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:19:49.726020    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd\": container with ID starting with aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd not found: ID does not exist" containerID="aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd"
Jul 17 06:19:49.726129 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.726056    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd"} err="failed to get container status \"aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd\": rpc error: code = NotFound desc = could not find container \"aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd\": container with ID starting with aebdde4bfdfb756178adb6e1581768ff1e1e11e3dd94d244f024db63ed9ad0dd not found: ID does not exist"
Jul 17 06:19:49.740397 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.740367    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"]
Jul 17 06:19:49.746026 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.745998    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3483d7/test-model-3483d7-kserve-865497fc79-jrhfh"]
Jul 17 06:19:49.752680 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.752639    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="434bdebd-f342-45c4-80da-a8a418e87854" path="/var/lib/kubelet/pods/434bdebd-f342-45c4-80da-a8a418e87854/volumes"
Jul 17 06:19:49.753095 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:49.753079    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d3572cb-7f1b-4939-92b0-ee55d730c490" path="/var/lib/kubelet/pods/7d3572cb-7f1b-4939-92b0-ee55d730c490/volumes"
Jul 17 06:19:55.337022 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:55.336987    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:55.437977 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:55.437940    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:59.652503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:59.652465    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:19:59.858679 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:19:59.858643    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:00.554928 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:00.554886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:00.654902 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:00.654835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:10.551977 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:10.551911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:15.072688 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:15.072653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:21.269407 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:21.269368    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:21.426055 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:21.426020    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:25.830469 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:25.830431    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:26.067466 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:26.067429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:26.443460 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:26.443427    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:26.801293 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:26.801258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:27.603880 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:27.603827    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:29.498209 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:29.498174    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:29.629371 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:29.629341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:29.767880 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:29.767778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:29.863320 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:29.863281    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:30.277948 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:30.277905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:30.379006 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:30.378969    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:34.013492 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:34.013464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:34.539239 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:34.539201    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:34.832492 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:34.832411    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:36.081379 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:36.081343    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:37.469546 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:37.469516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:37.560745 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:37.560715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:37.793421 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:37.793331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:37.994281 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:37.994227    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:38.367026 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.366987    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"]
Jul 17 06:20:38.367396 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.367381    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3572cb-7f1b-4939-92b0-ee55d730c490" containerName="main"
Jul 17 06:20:38.367474 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.367398    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3572cb-7f1b-4939-92b0-ee55d730c490" containerName="main"
Jul 17 06:20:38.367474 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.367417    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434bdebd-f342-45c4-80da-a8a418e87854" containerName="main"
Jul 17 06:20:38.367474 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.367423    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="434bdebd-f342-45c4-80da-a8a418e87854" containerName="main"
Jul 17 06:20:38.367634 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.367493    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="434bdebd-f342-45c4-80da-a8a418e87854" containerName="main"
Jul 17 06:20:38.367634 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.367502    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3572cb-7f1b-4939-92b0-ee55d730c490" containerName="main"
Jul 17 06:20:38.370670 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.370651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:20:38.373408 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.373377    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:20:38.373681 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.373663    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:20:38.379402 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.379377    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"]
Jul 17 06:20:38.404981 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.404943    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bd3217-2e16-4ab2-8dd5-6e8c04489857-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc\" (UID: \"88bd3217-2e16-4ab2-8dd5-6e8c04489857\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:20:38.461077 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.461036    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:38.505541 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.505496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bd3217-2e16-4ab2-8dd5-6e8c04489857-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc\" (UID: \"88bd3217-2e16-4ab2-8dd5-6e8c04489857\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:20:38.506325 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.506297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bd3217-2e16-4ab2-8dd5-6e8c04489857-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc\" (UID: \"88bd3217-2e16-4ab2-8dd5-6e8c04489857\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:20:38.562202 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.562157    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:38.684040 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.683942    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:20:38.829945 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.829916    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"]
Jul 17 06:20:38.832530 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:20:38.832501    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bd3217_2e16_4ab2_8dd5_6e8c04489857.slice/crio-5fa94dfbc05a7935905c46ec4be90a45d82c9719c75f800fc2bd575b54cc4da3 WatchSource:0}: Error finding container 5fa94dfbc05a7935905c46ec4be90a45d82c9719c75f800fc2bd575b54cc4da3: Status 404 returned error can't find the container with id 5fa94dfbc05a7935905c46ec4be90a45d82c9719c75f800fc2bd575b54cc4da3
Jul 17 06:20:38.834554 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.834538    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 06:20:38.952566 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:38.952477    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" event={"ID":"88bd3217-2e16-4ab2-8dd5-6e8c04489857","Type":"ContainerStarted","Data":"5fa94dfbc05a7935905c46ec4be90a45d82c9719c75f800fc2bd575b54cc4da3"}
Jul 17 06:20:41.955956 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:41.955916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:41.969937 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:41.969903    2559 generic.go:356] "Generic (PLEG): container finished" podID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerID="de26700b48788536015ba60d99858ba81ab1d53b1e1e9d96feaa0efb4739e531" exitCode=6
Jul 17 06:20:41.970138 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:41.969974    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" event={"ID":"88bd3217-2e16-4ab2-8dd5-6e8c04489857","Type":"ContainerDied","Data":"de26700b48788536015ba60d99858ba81ab1d53b1e1e9d96feaa0efb4739e531"}
Jul 17 06:20:41.970207 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:41.970190    2559 scope.go:117] "RemoveContainer" containerID="de26700b48788536015ba60d99858ba81ab1d53b1e1e9d96feaa0efb4739e531"
Jul 17 06:20:42.977254 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:42.977224    2559 generic.go:356] "Generic (PLEG): container finished" podID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerID="52993403cb76b023776d80a62e94750bb0a4ead7fe2c6191b4b36fb4eefa1302" exitCode=6
Jul 17 06:20:42.977749 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:42.977304    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" event={"ID":"88bd3217-2e16-4ab2-8dd5-6e8c04489857","Type":"ContainerDied","Data":"52993403cb76b023776d80a62e94750bb0a4ead7fe2c6191b4b36fb4eefa1302"}
Jul 17 06:20:42.977749 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:42.977359    2559 scope.go:117] "RemoveContainer" containerID="de26700b48788536015ba60d99858ba81ab1d53b1e1e9d96feaa0efb4739e531"
Jul 17 06:20:42.977749 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:42.977557    2559 scope.go:117] "RemoveContainer" containerID="52993403cb76b023776d80a62e94750bb0a4ead7fe2c6191b4b36fb4eefa1302"
Jul 17 06:20:42.977944 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:20:42.977820    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc_odh-ai-gateway-infra(88bd3217-2e16-4ab2-8dd5-6e8c04489857)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857"
Jul 17 06:20:43.983777 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:43.983751    2559 scope.go:117] "RemoveContainer" containerID="52993403cb76b023776d80a62e94750bb0a4ead7fe2c6191b4b36fb4eefa1302"
Jul 17 06:20:43.984217 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:20:43.983976    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc_odh-ai-gateway-infra(88bd3217-2e16-4ab2-8dd5-6e8c04489857)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857"
Jul 17 06:20:50.773492 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:50.773412    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:50.873528 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:50.873492    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:51.858614 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:51.858577    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:51.959559 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:51.959525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:20:52.656036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.656000    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"]
Jul 17 06:20:52.660757 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.660739    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:20:52.666831 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.666799    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"]
Jul 17 06:20:52.739336 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.739297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eef61eb-8dce-48e4-8470-840736e10e51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt\" (UID: \"7eef61eb-8dce-48e4-8470-840736e10e51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:20:52.839833 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.839789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eef61eb-8dce-48e4-8470-840736e10e51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt\" (UID: \"7eef61eb-8dce-48e4-8470-840736e10e51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:20:52.840468 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.840446    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eef61eb-8dce-48e4-8470-840736e10e51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt\" (UID: \"7eef61eb-8dce-48e4-8470-840736e10e51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:20:52.975193 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:52.975147    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:20:53.125210 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:53.121284    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"]
Jul 17 06:20:54.031474 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:54.031379    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eef61eb-8dce-48e4-8470-840736e10e51" containerID="8d786bdba1ba80144b90989b0588da479bfa945e0cc95f80fed290d080f42beb" exitCode=6
Jul 17 06:20:54.031844 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:54.031465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" event={"ID":"7eef61eb-8dce-48e4-8470-840736e10e51","Type":"ContainerDied","Data":"8d786bdba1ba80144b90989b0588da479bfa945e0cc95f80fed290d080f42beb"}
Jul 17 06:20:54.031844 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:54.031517    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" event={"ID":"7eef61eb-8dce-48e4-8470-840736e10e51","Type":"ContainerStarted","Data":"de3538c6c928bc4444799e7bc1846cb640041e719b71013406fe8d8a383f653e"}
Jul 17 06:20:54.031844 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:54.031641    2559 scope.go:117] "RemoveContainer" containerID="8d786bdba1ba80144b90989b0588da479bfa945e0cc95f80fed290d080f42beb"
Jul 17 06:20:55.038211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:55.038178    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eef61eb-8dce-48e4-8470-840736e10e51" containerID="bc87978774c7b1ab00c43f80683873f5240001273ee42e4bb9f41dc7acce77e6" exitCode=6
Jul 17 06:20:55.038656 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:55.038218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" event={"ID":"7eef61eb-8dce-48e4-8470-840736e10e51","Type":"ContainerDied","Data":"bc87978774c7b1ab00c43f80683873f5240001273ee42e4bb9f41dc7acce77e6"}
Jul 17 06:20:55.038656 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:55.038265    2559 scope.go:117] "RemoveContainer" containerID="8d786bdba1ba80144b90989b0588da479bfa945e0cc95f80fed290d080f42beb"
Jul 17 06:20:55.038656 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:55.038499    2559 scope.go:117] "RemoveContainer" containerID="bc87978774c7b1ab00c43f80683873f5240001273ee42e4bb9f41dc7acce77e6"
Jul 17 06:20:55.038802 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:20:55.038752    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt_odh-ai-gateway-infra(7eef61eb-8dce-48e4-8470-840736e10e51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" podUID="7eef61eb-8dce-48e4-8470-840736e10e51"
Jul 17 06:20:56.044314 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:56.044285    2559 scope.go:117] "RemoveContainer" containerID="bc87978774c7b1ab00c43f80683873f5240001273ee42e4bb9f41dc7acce77e6"
Jul 17 06:20:56.044709 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:20:56.044497    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt_odh-ai-gateway-infra(7eef61eb-8dce-48e4-8470-840736e10e51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" podUID="7eef61eb-8dce-48e4-8470-840736e10e51"
Jul 17 06:20:57.746528 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:57.746496    2559 scope.go:117] "RemoveContainer" containerID="52993403cb76b023776d80a62e94750bb0a4ead7fe2c6191b4b36fb4eefa1302"
Jul 17 06:20:58.056130 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:58.056034    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" event={"ID":"88bd3217-2e16-4ab2-8dd5-6e8c04489857","Type":"ContainerStarted","Data":"5f5b1cf204368cda75b184a8beb668a580e6e0319bf3f3a6e5c75796534dc673"}
Jul 17 06:20:58.071843 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:58.071790    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" podStartSLOduration=17.728765805 podStartE2EDuration="20.071771652s" totalImagesPullingTime="2.343005847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:20:58.071666798 +0000 UTC m=+2882.850149759" watchObservedRunningTime="2026-07-17 06:20:58.071771652 +0000 UTC m=+2882.850254613"
Jul 17 06:20:59.062919 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:59.062879    2559 generic.go:356] "Generic (PLEG): container finished" podID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerID="5f5b1cf204368cda75b184a8beb668a580e6e0319bf3f3a6e5c75796534dc673" exitCode=6
Jul 17 06:20:59.063389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:59.062945    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" event={"ID":"88bd3217-2e16-4ab2-8dd5-6e8c04489857","Type":"ContainerDied","Data":"5f5b1cf204368cda75b184a8beb668a580e6e0319bf3f3a6e5c75796534dc673"}
Jul 17 06:20:59.063389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:59.063040    2559 scope.go:117] "RemoveContainer" containerID="52993403cb76b023776d80a62e94750bb0a4ead7fe2c6191b4b36fb4eefa1302"
Jul 17 06:20:59.063389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:59.063357    2559 scope.go:117] "RemoveContainer" containerID="5f5b1cf204368cda75b184a8beb668a580e6e0319bf3f3a6e5c75796534dc673"
Jul 17 06:20:59.063596 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:20:59.063576    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc_odh-ai-gateway-infra(88bd3217-2e16-4ab2-8dd5-6e8c04489857)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857"
Jul 17 06:20:59.082387 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:59.082351    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"]
Jul 17 06:20:59.922135 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:20:59.922103    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:00.021540 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.021508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:00.092662 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.092620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:00.214060 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.214031    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:00.219195 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.219165    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:21:00.313763 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.313725    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bd3217-2e16-4ab2-8dd5-6e8c04489857-maas-api-service-ca\") pod \"88bd3217-2e16-4ab2-8dd5-6e8c04489857\" (UID: \"88bd3217-2e16-4ab2-8dd5-6e8c04489857\") "
Jul 17 06:21:00.314249 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.314221    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bd3217-2e16-4ab2-8dd5-6e8c04489857-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "88bd3217-2e16-4ab2-8dd5-6e8c04489857" (UID: "88bd3217-2e16-4ab2-8dd5-6e8c04489857"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:21:00.415261 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:00.415217    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bd3217-2e16-4ab2-8dd5-6e8c04489857-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:21:01.075056 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:01.075028    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"
Jul 17 06:21:01.075217 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:01.075070    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc" event={"ID":"88bd3217-2e16-4ab2-8dd5-6e8c04489857","Type":"ContainerDied","Data":"5fa94dfbc05a7935905c46ec4be90a45d82c9719c75f800fc2bd575b54cc4da3"}
Jul 17 06:21:01.075217 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:01.075132    2559 scope.go:117] "RemoveContainer" containerID="5f5b1cf204368cda75b184a8beb668a580e6e0319bf3f3a6e5c75796534dc673"
Jul 17 06:21:01.098752 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:01.098717    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"]
Jul 17 06:21:01.104894 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:01.104850    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-pgsmc"]
Jul 17 06:21:01.752259 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:01.752217    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" path="/var/lib/kubelet/pods/88bd3217-2e16-4ab2-8dd5-6e8c04489857/volumes"
Jul 17 06:21:05.186431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:05.186393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:07.746563 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:07.746523    2559 scope.go:117] "RemoveContainer" containerID="bc87978774c7b1ab00c43f80683873f5240001273ee42e4bb9f41dc7acce77e6"
Jul 17 06:21:08.113130 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:08.113091    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" event={"ID":"7eef61eb-8dce-48e4-8470-840736e10e51","Type":"ContainerStarted","Data":"b2eb0b2f5e6880b1f445e336c7ed4057d50f28435d0668acd088f457fffaa706"}
Jul 17 06:21:08.131169 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:08.131108    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" podStartSLOduration=16.131089902 podStartE2EDuration="16.131089902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:21:08.129275782 +0000 UTC m=+2892.907758739" watchObservedRunningTime="2026-07-17 06:21:08.131089902 +0000 UTC m=+2892.909572861"
Jul 17 06:21:09.119043 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:09.119006    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eef61eb-8dce-48e4-8470-840736e10e51" containerID="b2eb0b2f5e6880b1f445e336c7ed4057d50f28435d0668acd088f457fffaa706" exitCode=6
Jul 17 06:21:09.119529 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:09.119057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" event={"ID":"7eef61eb-8dce-48e4-8470-840736e10e51","Type":"ContainerDied","Data":"b2eb0b2f5e6880b1f445e336c7ed4057d50f28435d0668acd088f457fffaa706"}
Jul 17 06:21:09.119529 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:09.119117    2559 scope.go:117] "RemoveContainer" containerID="bc87978774c7b1ab00c43f80683873f5240001273ee42e4bb9f41dc7acce77e6"
Jul 17 06:21:09.119529 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:09.119444    2559 scope.go:117] "RemoveContainer" containerID="b2eb0b2f5e6880b1f445e336c7ed4057d50f28435d0668acd088f457fffaa706"
Jul 17 06:21:09.119716 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:09.119688    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt_odh-ai-gateway-infra(7eef61eb-8dce-48e4-8470-840736e10e51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" podUID="7eef61eb-8dce-48e4-8470-840736e10e51"
Jul 17 06:21:09.149966 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:09.149929    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"]
Jul 17 06:21:10.266143 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:10.266112    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:21:10.411485 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:10.411379    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eef61eb-8dce-48e4-8470-840736e10e51-maas-api-service-ca\") pod \"7eef61eb-8dce-48e4-8470-840736e10e51\" (UID: \"7eef61eb-8dce-48e4-8470-840736e10e51\") "
Jul 17 06:21:10.411966 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:10.411934    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eef61eb-8dce-48e4-8470-840736e10e51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7eef61eb-8dce-48e4-8470-840736e10e51" (UID: "7eef61eb-8dce-48e4-8470-840736e10e51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:21:10.513043 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:10.512987    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eef61eb-8dce-48e4-8470-840736e10e51-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:21:10.877662 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:10.877621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:10.975138 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:10.975098    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:11.058884 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.058819    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"]
Jul 17 06:21:11.059321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059298    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:11.059321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059323    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:11.059479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059338    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.059479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059346    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.059479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059354    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.059479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059362    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.059479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059395    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:11.059479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059403    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:11.059768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059488    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:11.059768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059500    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:11.059768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059511    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.059768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059519    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.059768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.059544    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:11.062606 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.062582    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:11.070849 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.070818    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"]
Jul 17 06:21:11.131782 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.131689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt" event={"ID":"7eef61eb-8dce-48e4-8470-840736e10e51","Type":"ContainerDied","Data":"de3538c6c928bc4444799e7bc1846cb640041e719b71013406fe8d8a383f653e"}
Jul 17 06:21:11.131782 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.131708    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"
Jul 17 06:21:11.131782 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.131749    2559 scope.go:117] "RemoveContainer" containerID="b2eb0b2f5e6880b1f445e336c7ed4057d50f28435d0668acd088f457fffaa706"
Jul 17 06:21:11.160627 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.160584    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"]
Jul 17 06:21:11.164892 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.164840    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vxjpt"]
Jul 17 06:21:11.219829 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.219794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c2b5f-cfca-46c8-9839-d70db53d97ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-p7wrr\" (UID: \"b30c2b5f-cfca-46c8-9839-d70db53d97ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:11.321229 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.321188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c2b5f-cfca-46c8-9839-d70db53d97ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-p7wrr\" (UID: \"b30c2b5f-cfca-46c8-9839-d70db53d97ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:11.321880 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.321832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c2b5f-cfca-46c8-9839-d70db53d97ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-p7wrr\" (UID: \"b30c2b5f-cfca-46c8-9839-d70db53d97ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:11.375093 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.375051    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:11.530539 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.530510    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"]
Jul 17 06:21:11.532290 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:21:11.532261    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30c2b5f_cfca_46c8_9839_d70db53d97ed.slice/crio-e69ec1b180a18877c431d1576e60bfbe0f27593302112faeebdf364d9303b091 WatchSource:0}: Error finding container e69ec1b180a18877c431d1576e60bfbe0f27593302112faeebdf364d9303b091: Status 404 returned error can't find the container with id e69ec1b180a18877c431d1576e60bfbe0f27593302112faeebdf364d9303b091
Jul 17 06:21:11.752140 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:11.752107    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" path="/var/lib/kubelet/pods/7eef61eb-8dce-48e4-8470-840736e10e51/volumes"
Jul 17 06:21:12.140262 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:12.140154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerStarted","Data":"51ab601268f3b9f92cf13f3e23329e82711ad77b48b201f662da3afcba9e4714"}
Jul 17 06:21:12.140262 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:12.140211    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerStarted","Data":"e69ec1b180a18877c431d1576e60bfbe0f27593302112faeebdf364d9303b091"}
Jul 17 06:21:12.157406 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:12.157348    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" podStartSLOduration=1.157327894 podStartE2EDuration="1.157327894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:21:12.157013882 +0000 UTC m=+2896.935496844" watchObservedRunningTime="2026-07-17 06:21:12.157327894 +0000 UTC m=+2896.935810854"
Jul 17 06:21:13.145972 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:13.145938    2559 generic.go:356] "Generic (PLEG): container finished" podID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerID="51ab601268f3b9f92cf13f3e23329e82711ad77b48b201f662da3afcba9e4714" exitCode=6
Jul 17 06:21:13.146572 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:13.146036    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerDied","Data":"51ab601268f3b9f92cf13f3e23329e82711ad77b48b201f662da3afcba9e4714"}
Jul 17 06:21:13.146572 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:13.146183    2559 scope.go:117] "RemoveContainer" containerID="51ab601268f3b9f92cf13f3e23329e82711ad77b48b201f662da3afcba9e4714"
Jul 17 06:21:14.153449 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:14.153348    2559 generic.go:356] "Generic (PLEG): container finished" podID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerID="394c3cdd5b8d5843ad4024d58f951cd430154be08a2c438af348fcab12370dee" exitCode=6
Jul 17 06:21:14.153449 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:14.153431    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerDied","Data":"394c3cdd5b8d5843ad4024d58f951cd430154be08a2c438af348fcab12370dee"}
Jul 17 06:21:14.153917 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:14.153487    2559 scope.go:117] "RemoveContainer" containerID="51ab601268f3b9f92cf13f3e23329e82711ad77b48b201f662da3afcba9e4714"
Jul 17 06:21:14.153917 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:14.153700    2559 scope.go:117] "RemoveContainer" containerID="394c3cdd5b8d5843ad4024d58f951cd430154be08a2c438af348fcab12370dee"
Jul 17 06:21:14.154003 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:14.153983    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-p7wrr_odh-ai-gateway-infra(b30c2b5f-cfca-46c8-9839-d70db53d97ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed"
Jul 17 06:21:15.159921 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:15.159893    2559 scope.go:117] "RemoveContainer" containerID="394c3cdd5b8d5843ad4024d58f951cd430154be08a2c438af348fcab12370dee"
Jul 17 06:21:15.160314 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:15.160103    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-p7wrr_odh-ai-gateway-infra(b30c2b5f-cfca-46c8-9839-d70db53d97ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed"
Jul 17 06:21:15.865304 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:15.865267    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:16.089200 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.089169    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:16.205172 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.205082    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"]
Jul 17 06:21:16.205566 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.205453    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:16.205566 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.205465    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:16.205566 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.205475    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:16.205566 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.205480    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eef61eb-8dce-48e4-8470-840736e10e51" containerName="revoke-keys"
Jul 17 06:21:16.205566 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.205533    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="88bd3217-2e16-4ab2-8dd5-6e8c04489857" containerName="revoke-keys"
Jul 17 06:21:16.216466 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.216431    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"]
Jul 17 06:21:16.216639 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.216541    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:16.263071 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.263021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eba1339-1ddb-4775-a96c-38f0019e0dcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm\" (UID: \"2eba1339-1ddb-4775-a96c-38f0019e0dcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:16.364670 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.364624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eba1339-1ddb-4775-a96c-38f0019e0dcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm\" (UID: \"2eba1339-1ddb-4775-a96c-38f0019e0dcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:16.365462 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.365436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eba1339-1ddb-4775-a96c-38f0019e0dcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm\" (UID: \"2eba1339-1ddb-4775-a96c-38f0019e0dcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:16.527831 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.527791    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:16.668955 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:16.668913    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"]
Jul 17 06:21:16.672714 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:21:16.672685    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eba1339_1ddb_4775_a96c_38f0019e0dcb.slice/crio-782311070b8e44d1df6e7b95340c41107ab53097dee8775e9a9dca8eece0868c WatchSource:0}: Error finding container 782311070b8e44d1df6e7b95340c41107ab53097dee8775e9a9dca8eece0868c: Status 404 returned error can't find the container with id 782311070b8e44d1df6e7b95340c41107ab53097dee8775e9a9dca8eece0868c
Jul 17 06:21:17.170927 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:17.170800    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm" event={"ID":"2eba1339-1ddb-4775-a96c-38f0019e0dcb","Type":"ContainerStarted","Data":"4ed13fd3688509afc6046ebe0e12718f15702b2f9ca9ad9d54956f62ed7001d7"}
Jul 17 06:21:17.170927 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:17.170872    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm" event={"ID":"2eba1339-1ddb-4775-a96c-38f0019e0dcb","Type":"ContainerStarted","Data":"782311070b8e44d1df6e7b95340c41107ab53097dee8775e9a9dca8eece0868c"}
Jul 17 06:21:17.191491 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:17.191434    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm" podStartSLOduration=1.191416517 podStartE2EDuration="1.191416517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:21:17.189946143 +0000 UTC m=+2901.968429105" watchObservedRunningTime="2026-07-17 06:21:17.191416517 +0000 UTC m=+2901.969899518"
Jul 17 06:21:18.177291 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:18.177254    2559 generic.go:356] "Generic (PLEG): container finished" podID="2eba1339-1ddb-4775-a96c-38f0019e0dcb" containerID="4ed13fd3688509afc6046ebe0e12718f15702b2f9ca9ad9d54956f62ed7001d7" exitCode=0
Jul 17 06:21:18.177750 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:18.177340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm" event={"ID":"2eba1339-1ddb-4775-a96c-38f0019e0dcb","Type":"ContainerDied","Data":"4ed13fd3688509afc6046ebe0e12718f15702b2f9ca9ad9d54956f62ed7001d7"}
Jul 17 06:21:19.316794 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:19.316761    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:19.392057 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:19.392024    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eba1339-1ddb-4775-a96c-38f0019e0dcb-maas-api-service-ca\") pod \"2eba1339-1ddb-4775-a96c-38f0019e0dcb\" (UID: \"2eba1339-1ddb-4775-a96c-38f0019e0dcb\") "
Jul 17 06:21:19.392468 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:19.392445    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eba1339-1ddb-4775-a96c-38f0019e0dcb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2eba1339-1ddb-4775-a96c-38f0019e0dcb" (UID: "2eba1339-1ddb-4775-a96c-38f0019e0dcb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:21:19.493225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:19.493188    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eba1339-1ddb-4775-a96c-38f0019e0dcb-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:21:20.189426 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:20.189333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm" event={"ID":"2eba1339-1ddb-4775-a96c-38f0019e0dcb","Type":"ContainerDied","Data":"782311070b8e44d1df6e7b95340c41107ab53097dee8775e9a9dca8eece0868c"}
Jul 17 06:21:20.189426 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:20.189390    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="782311070b8e44d1df6e7b95340c41107ab53097dee8775e9a9dca8eece0868c"
Jul 17 06:21:20.189426 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:20.189345    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm"
Jul 17 06:21:20.904608 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:20.904574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:24.359218 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.359183    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"]
Jul 17 06:21:24.359650 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.359562    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eba1339-1ddb-4775-a96c-38f0019e0dcb" containerName="revoke-keys"
Jul 17 06:21:24.359650 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.359575    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eba1339-1ddb-4775-a96c-38f0019e0dcb" containerName="revoke-keys"
Jul 17 06:21:24.359650 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.359634    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eba1339-1ddb-4775-a96c-38f0019e0dcb" containerName="revoke-keys"
Jul 17 06:21:24.361800 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.361782    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:24.369787 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.369757    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"]
Jul 17 06:21:24.435424 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.435384    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6\" (UID: \"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:24.535958 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.535919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6\" (UID: \"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:24.536615 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.536592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6\" (UID: \"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:24.674065 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.673971    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:24.812811 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:24.812782    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"]
Jul 17 06:21:24.818039 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:21:24.818002    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a9090d_6a23_415a_9e74_dc4d30d7dd5f.slice/crio-1e879ffefb778c3a6b412c459033f638347d1065b93b00b0b327ca1a77b0471a WatchSource:0}: Error finding container 1e879ffefb778c3a6b412c459033f638347d1065b93b00b0b327ca1a77b0471a: Status 404 returned error can't find the container with id 1e879ffefb778c3a6b412c459033f638347d1065b93b00b0b327ca1a77b0471a
Jul 17 06:21:25.214084 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:25.214046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerStarted","Data":"2e2bc3bdb4a9fd12a77a059377c4c93846d2c23b13dbe75383d2ad668eec5c61"}
Jul 17 06:21:25.214270 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:25.214104    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerStarted","Data":"1e879ffefb778c3a6b412c459033f638347d1065b93b00b0b327ca1a77b0471a"}
Jul 17 06:21:25.230627 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:25.230518    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" podStartSLOduration=1.230498501 podStartE2EDuration="1.230498501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:21:25.228700136 +0000 UTC m=+2910.007183097" watchObservedRunningTime="2026-07-17 06:21:25.230498501 +0000 UTC m=+2910.008981460"
Jul 17 06:21:26.222246 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:26.222211    2559 generic.go:356] "Generic (PLEG): container finished" podID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerID="2e2bc3bdb4a9fd12a77a059377c4c93846d2c23b13dbe75383d2ad668eec5c61" exitCode=6
Jul 17 06:21:26.222742 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:26.222304    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerDied","Data":"2e2bc3bdb4a9fd12a77a059377c4c93846d2c23b13dbe75383d2ad668eec5c61"}
Jul 17 06:21:26.222742 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:26.222568    2559 scope.go:117] "RemoveContainer" containerID="2e2bc3bdb4a9fd12a77a059377c4c93846d2c23b13dbe75383d2ad668eec5c61"
Jul 17 06:21:26.972443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:26.972405    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:27.228736 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:27.228704    2559 generic.go:356] "Generic (PLEG): container finished" podID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerID="32cef69393f2d0b51c724c09b1e76f6b128cfab61dbb0303d049ff8e5a5193fd" exitCode=6
Jul 17 06:21:27.229211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:27.228799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerDied","Data":"32cef69393f2d0b51c724c09b1e76f6b128cfab61dbb0303d049ff8e5a5193fd"}
Jul 17 06:21:27.229211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:27.228879    2559 scope.go:117] "RemoveContainer" containerID="2e2bc3bdb4a9fd12a77a059377c4c93846d2c23b13dbe75383d2ad668eec5c61"
Jul 17 06:21:27.229211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:27.229049    2559 scope.go:117] "RemoveContainer" containerID="32cef69393f2d0b51c724c09b1e76f6b128cfab61dbb0303d049ff8e5a5193fd"
Jul 17 06:21:27.229356 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:27.229258    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6_odh-ai-gateway-infra(f7a9090d-6a23-415a-9e74-dc4d30d7dd5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f"
Jul 17 06:21:28.236188 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:28.236158    2559 scope.go:117] "RemoveContainer" containerID="32cef69393f2d0b51c724c09b1e76f6b128cfab61dbb0303d049ff8e5a5193fd"
Jul 17 06:21:28.236579 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:28.236396    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6_odh-ai-gateway-infra(f7a9090d-6a23-415a-9e74-dc4d30d7dd5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f"
Jul 17 06:21:30.746765 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:30.746730    2559 scope.go:117] "RemoveContainer" containerID="394c3cdd5b8d5843ad4024d58f951cd430154be08a2c438af348fcab12370dee"
Jul 17 06:21:31.252785 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:31.252749    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerStarted","Data":"4fbbd38542f0c50afcdb34815e80d925e44683ec94e51d9d194c2862ff6d8652"}
Jul 17 06:21:31.303668 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:31.303633    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:32.260468 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:32.260431    2559 generic.go:356] "Generic (PLEG): container finished" podID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerID="4fbbd38542f0c50afcdb34815e80d925e44683ec94e51d9d194c2862ff6d8652" exitCode=6
Jul 17 06:21:32.261009 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:32.260508    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerDied","Data":"4fbbd38542f0c50afcdb34815e80d925e44683ec94e51d9d194c2862ff6d8652"}
Jul 17 06:21:32.261009 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:32.260589    2559 scope.go:117] "RemoveContainer" containerID="394c3cdd5b8d5843ad4024d58f951cd430154be08a2c438af348fcab12370dee"
Jul 17 06:21:32.261009 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:32.260841    2559 scope.go:117] "RemoveContainer" containerID="4fbbd38542f0c50afcdb34815e80d925e44683ec94e51d9d194c2862ff6d8652"
Jul 17 06:21:32.261165 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:32.261110    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-p7wrr_odh-ai-gateway-infra(b30c2b5f-cfca-46c8-9839-d70db53d97ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed"
Jul 17 06:21:32.278736 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:32.278708    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"]
Jul 17 06:21:33.410185 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:33.410158    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:33.518208 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:33.518109    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c2b5f-cfca-46c8-9839-d70db53d97ed-maas-api-service-ca\") pod \"b30c2b5f-cfca-46c8-9839-d70db53d97ed\" (UID: \"b30c2b5f-cfca-46c8-9839-d70db53d97ed\") "
Jul 17 06:21:33.518503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:33.518477    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30c2b5f-cfca-46c8-9839-d70db53d97ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b30c2b5f-cfca-46c8-9839-d70db53d97ed" (UID: "b30c2b5f-cfca-46c8-9839-d70db53d97ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:21:33.618770 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:33.618725    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c2b5f-cfca-46c8-9839-d70db53d97ed-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:21:34.273618 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:34.273581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr" event={"ID":"b30c2b5f-cfca-46c8-9839-d70db53d97ed","Type":"ContainerDied","Data":"e69ec1b180a18877c431d1576e60bfbe0f27593302112faeebdf364d9303b091"}
Jul 17 06:21:34.273836 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:34.273651    2559 scope.go:117] "RemoveContainer" containerID="4fbbd38542f0c50afcdb34815e80d925e44683ec94e51d9d194c2862ff6d8652"
Jul 17 06:21:34.273836 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:34.273588    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"
Jul 17 06:21:34.290011 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:34.289977    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"]
Jul 17 06:21:34.294083 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:34.294050    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-p7wrr"]
Jul 17 06:21:35.752192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:35.752158    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" path="/var/lib/kubelet/pods/b30c2b5f-cfca-46c8-9839-d70db53d97ed/volumes"
Jul 17 06:21:37.291575 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:37.291539    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:37.393160 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:37.393131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:39.746660 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:39.746624    2559 scope.go:117] "RemoveContainer" containerID="32cef69393f2d0b51c724c09b1e76f6b128cfab61dbb0303d049ff8e5a5193fd"
Jul 17 06:21:40.309760 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:40.309724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerStarted","Data":"f204c75cd8b166cb8707c154d95433f4fc81d1190a782da432128edc4e5f3a5a"}
Jul 17 06:21:41.317326 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:41.317292    2559 generic.go:356] "Generic (PLEG): container finished" podID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerID="f204c75cd8b166cb8707c154d95433f4fc81d1190a782da432128edc4e5f3a5a" exitCode=6
Jul 17 06:21:41.317814 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:41.317366    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerDied","Data":"f204c75cd8b166cb8707c154d95433f4fc81d1190a782da432128edc4e5f3a5a"}
Jul 17 06:21:41.317814 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:41.317466    2559 scope.go:117] "RemoveContainer" containerID="32cef69393f2d0b51c724c09b1e76f6b128cfab61dbb0303d049ff8e5a5193fd"
Jul 17 06:21:41.317814 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:41.317801    2559 scope.go:117] "RemoveContainer" containerID="f204c75cd8b166cb8707c154d95433f4fc81d1190a782da432128edc4e5f3a5a"
Jul 17 06:21:41.318057 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:21:41.318036    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6_odh-ai-gateway-infra(f7a9090d-6a23-415a-9e74-dc4d30d7dd5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f"
Jul 17 06:21:41.341127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:41.341089    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"]
Jul 17 06:21:41.993076 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:41.993047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:42.225638 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.225607    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:42.461997 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.461967    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:42.575358 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.575270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:42.600478 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.600446    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f-maas-api-service-ca\") pod \"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f\" (UID: \"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f\") "
Jul 17 06:21:42.600816 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.600791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" (UID: "f7a9090d-6a23-415a-9e74-dc4d30d7dd5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:21:42.701338 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.701299    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:21:42.890091 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:42.889993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:43.274159 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.274124    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:43.330346 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.330313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6" event={"ID":"f7a9090d-6a23-415a-9e74-dc4d30d7dd5f","Type":"ContainerDied","Data":"1e879ffefb778c3a6b412c459033f638347d1065b93b00b0b327ca1a77b0471a"}
Jul 17 06:21:43.330346 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.330337    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"
Jul 17 06:21:43.330579 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.330377    2559 scope.go:117] "RemoveContainer" containerID="f204c75cd8b166cb8707c154d95433f4fc81d1190a782da432128edc4e5f3a5a"
Jul 17 06:21:43.372850 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.372814    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"]
Jul 17 06:21:43.386339 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.386301    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-5fjm6"]
Jul 17 06:21:43.752079 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:43.752036    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" path="/var/lib/kubelet/pods/f7a9090d-6a23-415a-9e74-dc4d30d7dd5f/volumes"
Jul 17 06:21:45.785160 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:45.785108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:45.873997 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:45.873967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:45.946099 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:45.946069    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:46.046257 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:46.046167    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:46.460269 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:46.460177    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:46.560698 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:46.560662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:50.284909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:50.284851    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:50.648317 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:50.648218    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:50.942962 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:50.942879    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:51.102226 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:51.102194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:53.858430 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:53.858392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:53.940131 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:53.940098    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:54.004496 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:54.004460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:54.124011 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:54.123921    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:54.523658 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:54.523618    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:54.623852 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:54.623819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:21:58.328326 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:21:58.328285    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:07.870609 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:07.870576    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:07.971938 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:07.971905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:10.998391 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:10.998358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:11.090726 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.090693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:11.097372 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097340    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"]
Jul 17 06:22:11.097781 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097764    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097784    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097801    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097809    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097821    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097830    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097842    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097850    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097880    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.097888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097889    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.098328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097903    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.098328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097911    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.098328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.097995    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.098328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.098005    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.098328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.098016    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:11.098328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.098026    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:11.101496 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.101474    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:11.104249 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.104226    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:22:11.104370 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.104279    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:22:11.108094 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.108065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"]
Jul 17 06:22:11.153507 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.153470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fe9d7b1-2db2-4832-a603-b655abe6a071-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-q49z8\" (UID: \"2fe9d7b1-2db2-4832-a603-b655abe6a071\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:11.254307 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.254206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fe9d7b1-2db2-4832-a603-b655abe6a071-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-q49z8\" (UID: \"2fe9d7b1-2db2-4832-a603-b655abe6a071\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:11.254905 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.254848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fe9d7b1-2db2-4832-a603-b655abe6a071-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-q49z8\" (UID: \"2fe9d7b1-2db2-4832-a603-b655abe6a071\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:11.413964 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.413918    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:11.557044 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.557016    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"]
Jul 17 06:22:11.559297 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:22:11.559265    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe9d7b1_2db2_4832_a603_b655abe6a071.slice/crio-cfc2c4f75eab737e7ea7d14111e3ef241995b715674faba443bcc055d38de49a WatchSource:0}: Error finding container cfc2c4f75eab737e7ea7d14111e3ef241995b715674faba443bcc055d38de49a: Status 404 returned error can't find the container with id cfc2c4f75eab737e7ea7d14111e3ef241995b715674faba443bcc055d38de49a
Jul 17 06:22:11.723846 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:11.723808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:12.483363 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:12.483331    2559 generic.go:356] "Generic (PLEG): container finished" podID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerID="7ed04eca9327520a2937c184cb6fb16c74009494451e33c3c910d71f5a8155dd" exitCode=6
Jul 17 06:22:12.483837 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:12.483422    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" event={"ID":"2fe9d7b1-2db2-4832-a603-b655abe6a071","Type":"ContainerDied","Data":"7ed04eca9327520a2937c184cb6fb16c74009494451e33c3c910d71f5a8155dd"}
Jul 17 06:22:12.483837 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:12.483520    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" event={"ID":"2fe9d7b1-2db2-4832-a603-b655abe6a071","Type":"ContainerStarted","Data":"cfc2c4f75eab737e7ea7d14111e3ef241995b715674faba443bcc055d38de49a"}
Jul 17 06:22:12.483837 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:12.483631    2559 scope.go:117] "RemoveContainer" containerID="7ed04eca9327520a2937c184cb6fb16c74009494451e33c3c910d71f5a8155dd"
Jul 17 06:22:13.489475 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:13.489439    2559 generic.go:356] "Generic (PLEG): container finished" podID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerID="afdb10db3baf1841ff02971668986baf605a9f647da825e9aa184deead4c31d4" exitCode=6
Jul 17 06:22:13.490018 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:13.489528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" event={"ID":"2fe9d7b1-2db2-4832-a603-b655abe6a071","Type":"ContainerDied","Data":"afdb10db3baf1841ff02971668986baf605a9f647da825e9aa184deead4c31d4"}
Jul 17 06:22:13.490018 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:13.489589    2559 scope.go:117] "RemoveContainer" containerID="7ed04eca9327520a2937c184cb6fb16c74009494451e33c3c910d71f5a8155dd"
Jul 17 06:22:13.490018 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:13.489802    2559 scope.go:117] "RemoveContainer" containerID="afdb10db3baf1841ff02971668986baf605a9f647da825e9aa184deead4c31d4"
Jul 17 06:22:13.490169 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:13.490088    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-q49z8_odh-ai-gateway-infra(2fe9d7b1-2db2-4832-a603-b655abe6a071)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071"
Jul 17 06:22:14.402647 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.402605    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"]
Jul 17 06:22:14.403129 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.403111    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30c2b5f-cfca-46c8-9839-d70db53d97ed" containerName="revoke-keys"
Jul 17 06:22:14.403253 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.403134    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a9090d-6a23-415a-9e74-dc4d30d7dd5f" containerName="revoke-keys"
Jul 17 06:22:14.407408 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.407383    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:14.412303 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.412272    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"]
Jul 17 06:22:14.468401 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.468370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:14.486080 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.486034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3e8b87-be24-4087-a491-b751e4d2a57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-h79q9\" (UID: \"8b3e8b87-be24-4087-a491-b751e4d2a57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:14.495770 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.495743    2559 scope.go:117] "RemoveContainer" containerID="afdb10db3baf1841ff02971668986baf605a9f647da825e9aa184deead4c31d4"
Jul 17 06:22:14.496206 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:14.495995    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-q49z8_odh-ai-gateway-infra(2fe9d7b1-2db2-4832-a603-b655abe6a071)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071"
Jul 17 06:22:14.506536 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.506508    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"]
Jul 17 06:22:14.509981 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.509965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.513463 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.513436    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"kube-root-ca.crt\""
Jul 17 06:22:14.513622 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.513499    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"default-dockercfg-btm27\""
Jul 17 06:22:14.513698 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.513634    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"auth-test-model-e9bcaa-kserve-self-signed-certs\""
Jul 17 06:22:14.513807 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.513791    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"openshift-service-ca.crt\""
Jul 17 06:22:14.524724 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.524695    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"]
Jul 17 06:22:14.587367 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tls-certs\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.587576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv7zn\" (UniqueName: \"kubernetes.io/projected/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-kube-api-access-fv7zn\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.587576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3e8b87-be24-4087-a491-b751e4d2a57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-h79q9\" (UID: \"8b3e8b87-be24-4087-a491-b751e4d2a57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:14.587576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-dshm\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.587576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tmp-dir\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.587576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-model-cache\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.587771 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.587586    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-home\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.588208 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.588188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3e8b87-be24-4087-a491-b751e4d2a57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-h79q9\" (UID: \"8b3e8b87-be24-4087-a491-b751e4d2a57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:14.688378 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688269    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tmp-dir\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688570 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688389    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-model-cache\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688570 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688424    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-home\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688570 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tls-certs\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688570 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv7zn\" (UniqueName: \"kubernetes.io/projected/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-kube-api-access-fv7zn\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688570 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-dshm\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688892 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tmp-dir\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688892 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-model-cache\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.688892 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.688794    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-home\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.690881 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.690847    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-dshm\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.691209 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.691188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tls-certs\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.696366 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.696341    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv7zn\" (UniqueName: \"kubernetes.io/projected/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-kube-api-access-fv7zn\") pod \"auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") " pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.720008 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.719963    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:14.821318 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.821279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:14.871586 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.871516    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"]
Jul 17 06:22:14.875054 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:22:14.874988    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3e8b87_be24_4087_a491_b751e4d2a57e.slice/crio-9e86f6367ba47f713957a97ba7ff2a4b17e21b0b13fceec0f2c5dd659c7f7035 WatchSource:0}: Error finding container 9e86f6367ba47f713957a97ba7ff2a4b17e21b0b13fceec0f2c5dd659c7f7035: Status 404 returned error can't find the container with id 9e86f6367ba47f713957a97ba7ff2a4b17e21b0b13fceec0f2c5dd659c7f7035
Jul 17 06:22:14.975220 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:14.975188    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"]
Jul 17 06:22:14.978265 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:22:14.978240    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3d35527_5dfa_46a4_8e9b_8f6f408c18ac.slice/crio-e9f3b97bb0bb524409da1aa47627d98284ec118f4fca8845712679ecb8e35cbb WatchSource:0}: Error finding container e9f3b97bb0bb524409da1aa47627d98284ec118f4fca8845712679ecb8e35cbb: Status 404 returned error can't find the container with id e9f3b97bb0bb524409da1aa47627d98284ec118f4fca8845712679ecb8e35cbb
Jul 17 06:22:15.503982 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.503940    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf" event={"ID":"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac","Type":"ContainerStarted","Data":"bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd"}
Jul 17 06:22:15.504491 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.503990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf" event={"ID":"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac","Type":"ContainerStarted","Data":"e9f3b97bb0bb524409da1aa47627d98284ec118f4fca8845712679ecb8e35cbb"}
Jul 17 06:22:15.505504 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.505478    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerID="008a22e81643e7328835cc11abb1b6868150b3b26684efd3d0f9fed6fb3b7747" exitCode=6
Jul 17 06:22:15.505637 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.505560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" event={"ID":"8b3e8b87-be24-4087-a491-b751e4d2a57e","Type":"ContainerDied","Data":"008a22e81643e7328835cc11abb1b6868150b3b26684efd3d0f9fed6fb3b7747"}
Jul 17 06:22:15.505637 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.505613    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" event={"ID":"8b3e8b87-be24-4087-a491-b751e4d2a57e","Type":"ContainerStarted","Data":"9e86f6367ba47f713957a97ba7ff2a4b17e21b0b13fceec0f2c5dd659c7f7035"}
Jul 17 06:22:15.505717 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.505703    2559 scope.go:117] "RemoveContainer" containerID="008a22e81643e7328835cc11abb1b6868150b3b26684efd3d0f9fed6fb3b7747"
Jul 17 06:22:15.520950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:15.520904    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf" podStartSLOduration=1.520850399 podStartE2EDuration="1.520850399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:22:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:22:15.520110746 +0000 UTC m=+2960.298593706" watchObservedRunningTime="2026-07-17 06:22:15.520850399 +0000 UTC m=+2960.299333359"
Jul 17 06:22:16.364733 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:16.364693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:16.459545 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:16.459503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:16.511930 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:16.511891    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerID="d6824de63f7bc7cc85823ccca0d5b361875767abdef3785b55f685263ea5ac42" exitCode=6
Jul 17 06:22:16.512389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:16.511973    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" event={"ID":"8b3e8b87-be24-4087-a491-b751e4d2a57e","Type":"ContainerDied","Data":"d6824de63f7bc7cc85823ccca0d5b361875767abdef3785b55f685263ea5ac42"}
Jul 17 06:22:16.512389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:16.512055    2559 scope.go:117] "RemoveContainer" containerID="008a22e81643e7328835cc11abb1b6868150b3b26684efd3d0f9fed6fb3b7747"
Jul 17 06:22:16.512389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:16.512304    2559 scope.go:117] "RemoveContainer" containerID="d6824de63f7bc7cc85823ccca0d5b361875767abdef3785b55f685263ea5ac42"
Jul 17 06:22:16.512592 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:16.512553    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-h79q9_odh-ai-gateway-infra(8b3e8b87-be24-4087-a491-b751e4d2a57e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e"
Jul 17 06:22:17.519403 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:17.519328    2559 scope.go:117] "RemoveContainer" containerID="d6824de63f7bc7cc85823ccca0d5b361875767abdef3785b55f685263ea5ac42"
Jul 17 06:22:17.519817 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:17.519519    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-h79q9_odh-ai-gateway-infra(8b3e8b87-be24-4087-a491-b751e4d2a57e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e"
Jul 17 06:22:24.821632 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:24.821592    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:24.822162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:24.821654    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:24.834272 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:24.834241    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:25.568011 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:25.567979    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:22:29.746467 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:29.746430    2559 scope.go:117] "RemoveContainer" containerID="afdb10db3baf1841ff02971668986baf605a9f647da825e9aa184deead4c31d4"
Jul 17 06:22:30.581900 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:30.581843    2559 generic.go:356] "Generic (PLEG): container finished" podID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerID="64ef9540c84354f27ea9abcb723c01877132f29d1170432a1982f9d188479d2e" exitCode=6
Jul 17 06:22:30.582104 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:30.581913    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" event={"ID":"2fe9d7b1-2db2-4832-a603-b655abe6a071","Type":"ContainerDied","Data":"64ef9540c84354f27ea9abcb723c01877132f29d1170432a1982f9d188479d2e"}
Jul 17 06:22:30.582104 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:30.581979    2559 scope.go:117] "RemoveContainer" containerID="afdb10db3baf1841ff02971668986baf605a9f647da825e9aa184deead4c31d4"
Jul 17 06:22:30.582279 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:30.582263    2559 scope.go:117] "RemoveContainer" containerID="64ef9540c84354f27ea9abcb723c01877132f29d1170432a1982f9d188479d2e"
Jul 17 06:22:30.582513 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:30.582491    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-q49z8_odh-ai-gateway-infra(2fe9d7b1-2db2-4832-a603-b655abe6a071)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071"
Jul 17 06:22:30.746369 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:30.746331    2559 scope.go:117] "RemoveContainer" containerID="d6824de63f7bc7cc85823ccca0d5b361875767abdef3785b55f685263ea5ac42"
Jul 17 06:22:31.588463 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.588427    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerID="bb9c285a900181c4d16e934656a7e2d6556eb7968fb2b1a048b3c739292afc1e" exitCode=6
Jul 17 06:22:31.588941 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.588502    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" event={"ID":"8b3e8b87-be24-4087-a491-b751e4d2a57e","Type":"ContainerDied","Data":"bb9c285a900181c4d16e934656a7e2d6556eb7968fb2b1a048b3c739292afc1e"}
Jul 17 06:22:31.588941 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.588594    2559 scope.go:117] "RemoveContainer" containerID="d6824de63f7bc7cc85823ccca0d5b361875767abdef3785b55f685263ea5ac42"
Jul 17 06:22:31.588941 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.588883    2559 scope.go:117] "RemoveContainer" containerID="bb9c285a900181c4d16e934656a7e2d6556eb7968fb2b1a048b3c739292afc1e"
Jul 17 06:22:31.589200 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:31.589156    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-h79q9_odh-ai-gateway-infra(8b3e8b87-be24-4087-a491-b751e4d2a57e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e"
Jul 17 06:22:31.609902 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.609874    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"]
Jul 17 06:22:31.751654 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.751626    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:31.850463 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.850428    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fe9d7b1-2db2-4832-a603-b655abe6a071-maas-api-service-ca\") pod \"2fe9d7b1-2db2-4832-a603-b655abe6a071\" (UID: \"2fe9d7b1-2db2-4832-a603-b655abe6a071\") "
Jul 17 06:22:31.850817 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.850792    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fe9d7b1-2db2-4832-a603-b655abe6a071-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fe9d7b1-2db2-4832-a603-b655abe6a071" (UID: "2fe9d7b1-2db2-4832-a603-b655abe6a071"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:22:31.951921 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:31.951792    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fe9d7b1-2db2-4832-a603-b655abe6a071-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:22:32.597677 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.597641    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8" event={"ID":"2fe9d7b1-2db2-4832-a603-b655abe6a071","Type":"ContainerDied","Data":"cfc2c4f75eab737e7ea7d14111e3ef241995b715674faba443bcc055d38de49a"}
Jul 17 06:22:32.598087 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.597705    2559 scope.go:117] "RemoveContainer" containerID="64ef9540c84354f27ea9abcb723c01877132f29d1170432a1982f9d188479d2e"
Jul 17 06:22:32.598087 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.597655    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"
Jul 17 06:22:32.613428 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.613400    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"]
Jul 17 06:22:32.631079 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.631042    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"]
Jul 17 06:22:32.638410 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.638377    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-q49z8"]
Jul 17 06:22:32.755159 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.755128    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:32.860615 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.860516    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3e8b87-be24-4087-a491-b751e4d2a57e-maas-api-service-ca\") pod \"8b3e8b87-be24-4087-a491-b751e4d2a57e\" (UID: \"8b3e8b87-be24-4087-a491-b751e4d2a57e\") "
Jul 17 06:22:32.860953 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.860926    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b3e8b87-be24-4087-a491-b751e4d2a57e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b3e8b87-be24-4087-a491-b751e4d2a57e" (UID: "8b3e8b87-be24-4087-a491-b751e4d2a57e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:22:32.961657 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:32.961617    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3e8b87-be24-4087-a491-b751e4d2a57e-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:22:33.604562 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.604530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9" event={"ID":"8b3e8b87-be24-4087-a491-b751e4d2a57e","Type":"ContainerDied","Data":"9e86f6367ba47f713957a97ba7ff2a4b17e21b0b13fceec0f2c5dd659c7f7035"}
Jul 17 06:22:33.604562 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.604566    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"
Jul 17 06:22:33.605153 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.604587    2559 scope.go:117] "RemoveContainer" containerID="bb9c285a900181c4d16e934656a7e2d6556eb7968fb2b1a048b3c739292afc1e"
Jul 17 06:22:33.627914 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.627881    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"]
Jul 17 06:22:33.635873 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.635824    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-h79q9"]
Jul 17 06:22:33.752302 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.752263    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" path="/var/lib/kubelet/pods/2fe9d7b1-2db2-4832-a603-b655abe6a071/volumes"
Jul 17 06:22:33.752743 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:33.752720    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" path="/var/lib/kubelet/pods/8b3e8b87-be24-4087-a491-b751e4d2a57e/volumes"
Jul 17 06:22:35.359396 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:35.359354    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:42.745455 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745413    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"]
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745787    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745798    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745809    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745815    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745822    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745827    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745836    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745841    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745848    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745869    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745877    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.745909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745882    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.746607 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745944    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.746607 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745952    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:42.746607 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745958    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.746607 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.745966    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:42.749422 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.749396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:22:42.752483 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.752460    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:22:42.752609 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.752498    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:22:42.757848 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.757722    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"]
Jul 17 06:22:42.859729 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.859689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp\" (UID: \"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:22:42.961347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.961305    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp\" (UID: \"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:22:42.962002 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:42.961981    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp\" (UID: \"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:22:43.062498 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:43.062391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:22:43.213342 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:43.213314    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"]
Jul 17 06:22:43.215538 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:22:43.215506    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a0ffce_8d29_4a36_af5b_7a00d2b0d8fb.slice/crio-a42fff69cf5465ffb3d1e9d04d4c5462ca629f6f906f0a4c8a40da582b2a579b WatchSource:0}: Error finding container a42fff69cf5465ffb3d1e9d04d4c5462ca629f6f906f0a4c8a40da582b2a579b: Status 404 returned error can't find the container with id a42fff69cf5465ffb3d1e9d04d4c5462ca629f6f906f0a4c8a40da582b2a579b
Jul 17 06:22:43.661595 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:43.661486    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" event={"ID":"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb","Type":"ContainerStarted","Data":"b40b8608ed26d34b7a1289fdb5d207592523d554434467b5619f6a538ec1fa4b"}
Jul 17 06:22:43.661595 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:43.661545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" event={"ID":"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb","Type":"ContainerStarted","Data":"a42fff69cf5465ffb3d1e9d04d4c5462ca629f6f906f0a4c8a40da582b2a579b"}
Jul 17 06:22:43.679496 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:43.679442    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" podStartSLOduration=1.6794254469999998 podStartE2EDuration="1.679425447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:22:43.677992015 +0000 UTC m=+2988.456474974" watchObservedRunningTime="2026-07-17 06:22:43.679425447 +0000 UTC m=+2988.457908446"
Jul 17 06:22:44.668697 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:44.668660    2559 generic.go:356] "Generic (PLEG): container finished" podID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerID="b40b8608ed26d34b7a1289fdb5d207592523d554434467b5619f6a538ec1fa4b" exitCode=6
Jul 17 06:22:44.669303 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:44.668746    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" event={"ID":"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb","Type":"ContainerDied","Data":"b40b8608ed26d34b7a1289fdb5d207592523d554434467b5619f6a538ec1fa4b"}
Jul 17 06:22:44.669303 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:44.668985    2559 scope.go:117] "RemoveContainer" containerID="b40b8608ed26d34b7a1289fdb5d207592523d554434467b5619f6a538ec1fa4b"
Jul 17 06:22:45.676151 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:45.676034    2559 generic.go:356] "Generic (PLEG): container finished" podID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerID="dd7649e55317721af5e2e47919ce4ff00d2fade73b77b435e7b747dd89bb5a11" exitCode=6
Jul 17 06:22:45.676151 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:45.676111    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" event={"ID":"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb","Type":"ContainerDied","Data":"dd7649e55317721af5e2e47919ce4ff00d2fade73b77b435e7b747dd89bb5a11"}
Jul 17 06:22:45.676688 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:45.676179    2559 scope.go:117] "RemoveContainer" containerID="b40b8608ed26d34b7a1289fdb5d207592523d554434467b5619f6a538ec1fa4b"
Jul 17 06:22:45.676688 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:45.676391    2559 scope.go:117] "RemoveContainer" containerID="dd7649e55317721af5e2e47919ce4ff00d2fade73b77b435e7b747dd89bb5a11"
Jul 17 06:22:45.676688 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:45.676667    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp_odh-ai-gateway-infra(67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb"
Jul 17 06:22:46.683162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:46.683133    2559 scope.go:117] "RemoveContainer" containerID="dd7649e55317721af5e2e47919ce4ff00d2fade73b77b435e7b747dd89bb5a11"
Jul 17 06:22:46.683587 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:22:46.683337    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp_odh-ai-gateway-infra(67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb"
Jul 17 06:22:50.409224 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.409181    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"]
Jul 17 06:22:50.409795 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.409773    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe9d7b1-2db2-4832-a603-b655abe6a071" containerName="revoke-keys"
Jul 17 06:22:50.409918 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.409802    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b3e8b87-be24-4087-a491-b751e4d2a57e" containerName="revoke-keys"
Jul 17 06:22:50.414179 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.414151    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.418217 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.418189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-efbd92\"/\"auth-test-model-efbd92-kserve-self-signed-certs\""
Jul 17 06:22:50.418380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.418189    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-efbd92\"/\"kube-root-ca.crt\""
Jul 17 06:22:50.418380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.418189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-efbd92\"/\"default-dockercfg-vs94t\""
Jul 17 06:22:50.418380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.418226    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-efbd92\"/\"openshift-service-ca.crt\""
Jul 17 06:22:50.424696 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.424666    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"]
Jul 17 06:22:50.424987 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.424963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-model-cache\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.425137 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.425086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lb5x\" (UniqueName: \"kubernetes.io/projected/7088e3f6-35c1-4d92-95d9-e887d5845c32-kube-api-access-5lb5x\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.425212 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.425144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-home\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.425212 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.425179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-tmp-dir\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.425212 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.425206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088e3f6-35c1-4d92-95d9-e887d5845c32-tls-certs\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.425380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.425356    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-dshm\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.453037 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.453001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:50.526575 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.526534    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lb5x\" (UniqueName: \"kubernetes.io/projected/7088e3f6-35c1-4d92-95d9-e887d5845c32-kube-api-access-5lb5x\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.526575 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.526579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-home\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.526901 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.526602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-tmp-dir\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.526901 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.526621    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088e3f6-35c1-4d92-95d9-e887d5845c32-tls-certs\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.526901 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.526650    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-dshm\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.526901 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.526672    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-model-cache\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.527156 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.527073    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-tmp-dir\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.527156 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.527126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-home\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.527239 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.527133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-model-cache\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.529062 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.529045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-dshm\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.529383 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.529366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088e3f6-35c1-4d92-95d9-e887d5845c32-tls-certs\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.535228 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.535201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lb5x\" (UniqueName: \"kubernetes.io/projected/7088e3f6-35c1-4d92-95d9-e887d5845c32-kube-api-access-5lb5x\") pod \"auth-test-model-efbd92-kserve-cdd878db7-6q2zm\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") " pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.724935 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.724902    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:22:50.866378 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:50.866349    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"]
Jul 17 06:22:50.868984 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:22:50.868958    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7088e3f6_35c1_4d92_95d9_e887d5845c32.slice/crio-5c3eeb16b60f2fef2f0e70b66f606510db73e2d9c468398700f198010c71fd76 WatchSource:0}: Error finding container 5c3eeb16b60f2fef2f0e70b66f606510db73e2d9c468398700f198010c71fd76: Status 404 returned error can't find the container with id 5c3eeb16b60f2fef2f0e70b66f606510db73e2d9c468398700f198010c71fd76
Jul 17 06:22:51.633009 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:51.632976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:22:51.708552 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:51.708518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm" event={"ID":"7088e3f6-35c1-4d92-95d9-e887d5845c32","Type":"ContainerStarted","Data":"1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4"}
Jul 17 06:22:51.708764 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:51.708561    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm" event={"ID":"7088e3f6-35c1-4d92-95d9-e887d5845c32","Type":"ContainerStarted","Data":"5c3eeb16b60f2fef2f0e70b66f606510db73e2d9c468398700f198010c71fd76"}
Jul 17 06:22:51.726217 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:22:51.726160    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm" podStartSLOduration=1.7261411199999999 podStartE2EDuration="1.72614112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:22:51.725051999 +0000 UTC m=+2996.503534962" watchObservedRunningTime="2026-07-17 06:22:51.72614112 +0000 UTC m=+2996.504624090"
Jul 17 06:23:00.725054 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:00.725020    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:23:00.725524 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:00.725392    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:23:00.738182 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:00.738154    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:23:00.746753 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:00.746729    2559 scope.go:117] "RemoveContainer" containerID="dd7649e55317721af5e2e47919ce4ff00d2fade73b77b435e7b747dd89bb5a11"
Jul 17 06:23:00.765813 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:00.765786    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:23:01.759736 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:01.759637    2559 generic.go:356] "Generic (PLEG): container finished" podID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerID="a262336692d9928c9db585533f048713e3eeb6d841e3404c74edd19356806bf4" exitCode=6
Jul 17 06:23:01.759736 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:01.759710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" event={"ID":"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb","Type":"ContainerDied","Data":"a262336692d9928c9db585533f048713e3eeb6d841e3404c74edd19356806bf4"}
Jul 17 06:23:01.760339 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:01.759785    2559 scope.go:117] "RemoveContainer" containerID="dd7649e55317721af5e2e47919ce4ff00d2fade73b77b435e7b747dd89bb5a11"
Jul 17 06:23:01.760339 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:01.760200    2559 scope.go:117] "RemoveContainer" containerID="a262336692d9928c9db585533f048713e3eeb6d841e3404c74edd19356806bf4"
Jul 17 06:23:01.760450 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:23:01.760431    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp_odh-ai-gateway-infra(67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb"
Jul 17 06:23:02.788003 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:02.787966    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"]
Jul 17 06:23:02.928137 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:02.928109    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:23:03.039096 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.038997    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb-maas-api-service-ca\") pod \"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb\" (UID: \"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb\") "
Jul 17 06:23:03.039362 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.039339    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" (UID: "67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:23:03.139978 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.139942    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:23:03.773011 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.772985    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"
Jul 17 06:23:03.773173 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.773028    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp" event={"ID":"67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb","Type":"ContainerDied","Data":"a42fff69cf5465ffb3d1e9d04d4c5462ca629f6f906f0a4c8a40da582b2a579b"}
Jul 17 06:23:03.773173 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.773089    2559 scope.go:117] "RemoveContainer" containerID="a262336692d9928c9db585533f048713e3eeb6d841e3404c74edd19356806bf4"
Jul 17 06:23:03.789034 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.788997    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"]
Jul 17 06:23:03.793705 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:03.793676    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vd9pp"]
Jul 17 06:23:05.752774 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:23:05.752725    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" path="/var/lib/kubelet/pods/67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb/volumes"
Jul 17 06:25:01.007562 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:01.007526    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"]
Jul 17 06:25:01.074801 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:01.074768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:25:01.932988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:01.932954    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"]
Jul 17 06:25:01.950652 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:01.950575    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:25:16.027984 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.027938    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf" podUID="f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" containerName="main" containerID="cri-o://bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd" gracePeriod=45
Jul 17 06:25:16.285637 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.285565    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:25:16.389885 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.389818    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv7zn\" (UniqueName: \"kubernetes.io/projected/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-kube-api-access-fv7zn\") pod \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") "
Jul 17 06:25:16.390062 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.389923    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-home\") pod \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") "
Jul 17 06:25:16.390157 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390077    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tls-certs\") pod \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") "
Jul 17 06:25:16.390157 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390122    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-dshm\") pod \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") "
Jul 17 06:25:16.390157 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390155    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-model-cache\") pod \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") "
Jul 17 06:25:16.390321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390159    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-home" (OuterVolumeSpecName: "home") pod "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" (UID: "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:16.390321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390185    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tmp-dir\") pod \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\" (UID: \"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac\") "
Jul 17 06:25:16.390437 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390378    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" (UID: "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:16.390437 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390409    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" (UID: "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:16.390576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390527    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:16.390576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390551    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:16.390576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.390570    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:16.392318 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.392278    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-dshm" (OuterVolumeSpecName: "dshm") pod "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" (UID: "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:16.392318 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.392303    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-kube-api-access-fv7zn" (OuterVolumeSpecName: "kube-api-access-fv7zn") pod "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" (UID: "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac"). InnerVolumeSpecName "kube-api-access-fv7zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:25:16.392476 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.392317    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" (UID: "f3d35527-5dfa-46a4-8e9b-8f6f408c18ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:25:16.425355 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.425324    2559 generic.go:356] "Generic (PLEG): container finished" podID="f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" containerID="bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd" exitCode=0
Jul 17 06:25:16.425539 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.425366    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"
Jul 17 06:25:16.425539 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.425407    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf" event={"ID":"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac","Type":"ContainerDied","Data":"bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd"}
Jul 17 06:25:16.425539 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.425451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf" event={"ID":"f3d35527-5dfa-46a4-8e9b-8f6f408c18ac","Type":"ContainerDied","Data":"e9f3b97bb0bb524409da1aa47627d98284ec118f4fca8845712679ecb8e35cbb"}
Jul 17 06:25:16.425539 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.425469    2559 scope.go:117] "RemoveContainer" containerID="bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd"
Jul 17 06:25:16.434804 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.434785    2559 scope.go:117] "RemoveContainer" containerID="bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd"
Jul 17 06:25:16.435088 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:16.435069    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd\": container with ID starting with bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd not found: ID does not exist" containerID="bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd"
Jul 17 06:25:16.435137 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.435097    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd"} err="failed to get container status \"bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd\": rpc error: code = NotFound desc = could not find container \"bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd\": container with ID starting with bb153cd74c3c82079d21026d4ee9220f600972d4547bb988238843a3fa755ddd not found: ID does not exist"
Jul 17 06:25:16.448611 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.448579    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"]
Jul 17 06:25:16.453057 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.453021    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/auth-test-model-e9bcaa-kserve-5987b7846c-hh9zf"]
Jul 17 06:25:16.492060 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.492019    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv7zn\" (UniqueName: \"kubernetes.io/projected/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-kube-api-access-fv7zn\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:16.492060 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.492051    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:16.492060 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.492062    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:16.953953 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:16.953908    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm" podUID="7088e3f6-35c1-4d92-95d9-e887d5845c32" containerName="main" containerID="cri-o://1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4" gracePeriod=45
Jul 17 06:25:17.213616 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.213593    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:25:17.300314 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300277    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-home\") pod \"7088e3f6-35c1-4d92-95d9-e887d5845c32\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") "
Jul 17 06:25:17.300314 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300316    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-dshm\") pod \"7088e3f6-35c1-4d92-95d9-e887d5845c32\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") "
Jul 17 06:25:17.300608 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300374    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-model-cache\") pod \"7088e3f6-35c1-4d92-95d9-e887d5845c32\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") "
Jul 17 06:25:17.300608 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300412    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088e3f6-35c1-4d92-95d9-e887d5845c32-tls-certs\") pod \"7088e3f6-35c1-4d92-95d9-e887d5845c32\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") "
Jul 17 06:25:17.300608 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300444    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-tmp-dir\") pod \"7088e3f6-35c1-4d92-95d9-e887d5845c32\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") "
Jul 17 06:25:17.300608 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300498    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lb5x\" (UniqueName: \"kubernetes.io/projected/7088e3f6-35c1-4d92-95d9-e887d5845c32-kube-api-access-5lb5x\") pod \"7088e3f6-35c1-4d92-95d9-e887d5845c32\" (UID: \"7088e3f6-35c1-4d92-95d9-e887d5845c32\") "
Jul 17 06:25:17.300820 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300692    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-model-cache" (OuterVolumeSpecName: "model-cache") pod "7088e3f6-35c1-4d92-95d9-e887d5845c32" (UID: "7088e3f6-35c1-4d92-95d9-e887d5845c32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:17.300901 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300835    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:17.300963 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.300935    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-home" (OuterVolumeSpecName: "home") pod "7088e3f6-35c1-4d92-95d9-e887d5845c32" (UID: "7088e3f6-35c1-4d92-95d9-e887d5845c32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:17.301192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.301167    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7088e3f6-35c1-4d92-95d9-e887d5845c32" (UID: "7088e3f6-35c1-4d92-95d9-e887d5845c32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:17.302794 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.302772    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-dshm" (OuterVolumeSpecName: "dshm") pod "7088e3f6-35c1-4d92-95d9-e887d5845c32" (UID: "7088e3f6-35c1-4d92-95d9-e887d5845c32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:25:17.302886 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.302840    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7088e3f6-35c1-4d92-95d9-e887d5845c32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7088e3f6-35c1-4d92-95d9-e887d5845c32" (UID: "7088e3f6-35c1-4d92-95d9-e887d5845c32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:25:17.303006 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.302986    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7088e3f6-35c1-4d92-95d9-e887d5845c32-kube-api-access-5lb5x" (OuterVolumeSpecName: "kube-api-access-5lb5x") pod "7088e3f6-35c1-4d92-95d9-e887d5845c32" (UID: "7088e3f6-35c1-4d92-95d9-e887d5845c32"). InnerVolumeSpecName "kube-api-access-5lb5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:25:17.401780 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.401736    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:17.401780 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.401779    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lb5x\" (UniqueName: \"kubernetes.io/projected/7088e3f6-35c1-4d92-95d9-e887d5845c32-kube-api-access-5lb5x\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:17.402036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.401792    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:17.402036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.401803    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088e3f6-35c1-4d92-95d9-e887d5845c32-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:17.402036 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.401814    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088e3f6-35c1-4d92-95d9-e887d5845c32-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:17.433634 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.433595    2559 generic.go:356] "Generic (PLEG): container finished" podID="7088e3f6-35c1-4d92-95d9-e887d5845c32" containerID="1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4" exitCode=0
Jul 17 06:25:17.433820 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.433645    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"
Jul 17 06:25:17.433820 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.433683    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm" event={"ID":"7088e3f6-35c1-4d92-95d9-e887d5845c32","Type":"ContainerDied","Data":"1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4"}
Jul 17 06:25:17.433820 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.433734    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm" event={"ID":"7088e3f6-35c1-4d92-95d9-e887d5845c32","Type":"ContainerDied","Data":"5c3eeb16b60f2fef2f0e70b66f606510db73e2d9c468398700f198010c71fd76"}
Jul 17 06:25:17.433820 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.433756    2559 scope.go:117] "RemoveContainer" containerID="1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4"
Jul 17 06:25:17.442888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.442848    2559 scope.go:117] "RemoveContainer" containerID="1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4"
Jul 17 06:25:17.443215 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:17.443189    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4\": container with ID starting with 1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4 not found: ID does not exist" containerID="1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4"
Jul 17 06:25:17.443312 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.443226    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4"} err="failed to get container status \"1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4\": rpc error: code = NotFound desc = could not find container \"1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4\": container with ID starting with 1e560237aee27e54e4ce33ab29821f40e77b66c130577f5f59ab2157f2b087a4 not found: ID does not exist"
Jul 17 06:25:17.456134 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.456101    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"]
Jul 17 06:25:17.459885 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.459840    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-efbd92/auth-test-model-efbd92-kserve-cdd878db7-6q2zm"]
Jul 17 06:25:17.752434 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.752397    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7088e3f6-35c1-4d92-95d9-e887d5845c32" path="/var/lib/kubelet/pods/7088e3f6-35c1-4d92-95d9-e887d5845c32/volumes"
Jul 17 06:25:17.752818 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:17.752806    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" path="/var/lib/kubelet/pods/f3d35527-5dfa-46a4-8e9b-8f6f408c18ac/volumes"
Jul 17 06:25:18.599048 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599015    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"]
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599361    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599373    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599388    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599394    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599401    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" containerName="main"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599407    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" containerName="main"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599416    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7088e3f6-35c1-4d92-95d9-e887d5845c32" containerName="main"
Jul 17 06:25:18.599443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599421    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7088e3f6-35c1-4d92-95d9-e887d5845c32" containerName="main"
Jul 17 06:25:18.599730 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599478    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:18.599730 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599486    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:18.599730 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599494    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7088e3f6-35c1-4d92-95d9-e887d5845c32" containerName="main"
Jul 17 06:25:18.599730 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.599500    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d35527-5dfa-46a4-8e9b-8f6f408c18ac" containerName="main"
Jul 17 06:25:18.603482 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.603452    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:18.606443 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.606416    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:25:18.606583 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.606416    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:25:18.610824 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.610795    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"]
Jul 17 06:25:18.713575 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.713535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw\" (UID: \"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:18.814256 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.814214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw\" (UID: \"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:18.814928 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.814906    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw\" (UID: \"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:18.917193 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:18.917082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:19.067218 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:19.067172    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"]
Jul 17 06:25:19.075614 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:25:19.075585    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3adc9e_8bf9_4dd4_8282_04ea81c21ed4.slice/crio-8f6d4c87dfc4f7b32afa69ed679a5fe59fea643bd5c168e6aec7e425bbec8136 WatchSource:0}: Error finding container 8f6d4c87dfc4f7b32afa69ed679a5fe59fea643bd5c168e6aec7e425bbec8136: Status 404 returned error can't find the container with id 8f6d4c87dfc4f7b32afa69ed679a5fe59fea643bd5c168e6aec7e425bbec8136
Jul 17 06:25:19.446122 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:19.446087    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" event={"ID":"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4","Type":"ContainerStarted","Data":"f53788c7741094d2df77bc46b3668b725c6db781fa1d9e96758accc1e67aa713"}
Jul 17 06:25:19.446301 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:19.446144    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" event={"ID":"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4","Type":"ContainerStarted","Data":"8f6d4c87dfc4f7b32afa69ed679a5fe59fea643bd5c168e6aec7e425bbec8136"}
Jul 17 06:25:19.461879 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:19.461802    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" podStartSLOduration=1.461785303 podStartE2EDuration="1.461785303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:25:19.461003484 +0000 UTC m=+3144.239486445" watchObservedRunningTime="2026-07-17 06:25:19.461785303 +0000 UTC m=+3144.240268305"
Jul 17 06:25:20.452762 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:20.452728    2559 generic.go:356] "Generic (PLEG): container finished" podID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerID="f53788c7741094d2df77bc46b3668b725c6db781fa1d9e96758accc1e67aa713" exitCode=6
Jul 17 06:25:20.453224 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:20.452829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" event={"ID":"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4","Type":"ContainerDied","Data":"f53788c7741094d2df77bc46b3668b725c6db781fa1d9e96758accc1e67aa713"}
Jul 17 06:25:20.453224 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:20.453013    2559 scope.go:117] "RemoveContainer" containerID="f53788c7741094d2df77bc46b3668b725c6db781fa1d9e96758accc1e67aa713"
Jul 17 06:25:21.459403 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:21.459301    2559 generic.go:356] "Generic (PLEG): container finished" podID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerID="ce6aaddb4834bf5a0572fb01c4baa4712be15c576a26ef6efbe4f746f1e418ae" exitCode=6
Jul 17 06:25:21.459403 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:21.459339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" event={"ID":"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4","Type":"ContainerDied","Data":"ce6aaddb4834bf5a0572fb01c4baa4712be15c576a26ef6efbe4f746f1e418ae"}
Jul 17 06:25:21.459403 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:21.459389    2559 scope.go:117] "RemoveContainer" containerID="f53788c7741094d2df77bc46b3668b725c6db781fa1d9e96758accc1e67aa713"
Jul 17 06:25:21.459945 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:21.459625    2559 scope.go:117] "RemoveContainer" containerID="ce6aaddb4834bf5a0572fb01c4baa4712be15c576a26ef6efbe4f746f1e418ae"
Jul 17 06:25:21.459945 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:21.459908    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw_odh-ai-gateway-infra(5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4"
Jul 17 06:25:22.468327 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.468297    2559 scope.go:117] "RemoveContainer" containerID="ce6aaddb4834bf5a0572fb01c4baa4712be15c576a26ef6efbe4f746f1e418ae"
Jul 17 06:25:22.468711 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:22.468483    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw_odh-ai-gateway-infra(5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4"
Jul 17 06:25:22.928101 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.928002    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"]
Jul 17 06:25:22.928373 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.928359    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:22.928413 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.928376    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:22.928463 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.928454    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a0ffce-8d29-4a36-af5b-7a00d2b0d8fb" containerName="revoke-keys"
Jul 17 06:25:22.932806 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.932787    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:22.937991 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:22.937946    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"]
Jul 17 06:25:23.053832 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.053788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f96911-334e-49d7-bd92-258ca8b5c2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx\" (UID: \"d7f96911-334e-49d7-bd92-258ca8b5c2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:23.155156 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.155116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f96911-334e-49d7-bd92-258ca8b5c2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx\" (UID: \"d7f96911-334e-49d7-bd92-258ca8b5c2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:23.155776 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.155753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f96911-334e-49d7-bd92-258ca8b5c2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx\" (UID: \"d7f96911-334e-49d7-bd92-258ca8b5c2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:23.229838 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.229800    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"]
Jul 17 06:25:23.233416 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.233396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.236330 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.236306    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"openshift-service-ca.crt\""
Jul 17 06:25:23.237569 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.237542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"default-dockercfg-btm27\""
Jul 17 06:25:23.237690 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.237582    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"kube-root-ca.crt\""
Jul 17 06:25:23.237690 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.237550    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"sub-test-model-e9bcaa-kserve-self-signed-certs\""
Jul 17 06:25:23.244240 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.244215    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"]
Jul 17 06:25:23.245282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.245260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:23.257315 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.257256    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:25:23.357618 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.357574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-tmp-dir\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.357618 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.357619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9hlx\" (UniqueName: \"kubernetes.io/projected/d8c527b3-f523-4abc-abd7-157dae79bd7d-kube-api-access-f9hlx\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.357919 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.357641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-dshm\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.357919 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.357691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-model-cache\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.357919 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.357759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8c527b3-f523-4abc-abd7-157dae79bd7d-tls-certs\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.357919 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.357803    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-home\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.385436 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.385402    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"]
Jul 17 06:25:23.390138 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:25:23.390103    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f96911_334e_49d7_bd92_258ca8b5c2a8.slice/crio-e444d56efe572bd4a8f06a6f2633ce411488eedb15664fc75a3e67d858bdec41 WatchSource:0}: Error finding container e444d56efe572bd4a8f06a6f2633ce411488eedb15664fc75a3e67d858bdec41: Status 404 returned error can't find the container with id e444d56efe572bd4a8f06a6f2633ce411488eedb15664fc75a3e67d858bdec41
Jul 17 06:25:23.458714 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.458668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8c527b3-f523-4abc-abd7-157dae79bd7d-tls-certs\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.458714 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.458710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-home\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.458792    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-tmp-dir\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.458809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9hlx\" (UniqueName: \"kubernetes.io/projected/d8c527b3-f523-4abc-abd7-157dae79bd7d-kube-api-access-f9hlx\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.458829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-dshm\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.458846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-model-cache\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459307 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.459278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-home\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.459323    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-model-cache\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.459431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.459346    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-tmp-dir\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.461187 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.461164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-dshm\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.461486 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.461468    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8c527b3-f523-4abc-abd7-157dae79bd7d-tls-certs\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.467842 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.467814    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9hlx\" (UniqueName: \"kubernetes.io/projected/d8c527b3-f523-4abc-abd7-157dae79bd7d-kube-api-access-f9hlx\") pod \"sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") " pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.480940 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.480764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" event={"ID":"d7f96911-334e-49d7-bd92-258ca8b5c2a8","Type":"ContainerStarted","Data":"e444d56efe572bd4a8f06a6f2633ce411488eedb15664fc75a3e67d858bdec41"}
Jul 17 06:25:23.543252 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.543211    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:23.693126 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:23.693097    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"]
Jul 17 06:25:23.695608 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:25:23.695582    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8c527b3_f523_4abc_abd7_157dae79bd7d.slice/crio-fb7b1712818a108a21aa5c80c4cfa07f746904dda0cba1a03ace0ee89845a484 WatchSource:0}: Error finding container fb7b1712818a108a21aa5c80c4cfa07f746904dda0cba1a03ace0ee89845a484: Status 404 returned error can't find the container with id fb7b1712818a108a21aa5c80c4cfa07f746904dda0cba1a03ace0ee89845a484
Jul 17 06:25:24.487144 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:24.487107    2559 generic.go:356] "Generic (PLEG): container finished" podID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerID="4ffaf6ec58a7ee60e29b6b2b9c5ed2ed06c85e00f44be4db127be1081c5fdbbd" exitCode=6
Jul 17 06:25:24.487639 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:24.487193    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" event={"ID":"d7f96911-334e-49d7-bd92-258ca8b5c2a8","Type":"ContainerDied","Data":"4ffaf6ec58a7ee60e29b6b2b9c5ed2ed06c85e00f44be4db127be1081c5fdbbd"}
Jul 17 06:25:24.487639 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:24.487427    2559 scope.go:117] "RemoveContainer" containerID="4ffaf6ec58a7ee60e29b6b2b9c5ed2ed06c85e00f44be4db127be1081c5fdbbd"
Jul 17 06:25:24.489030 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:24.489004    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99" event={"ID":"d8c527b3-f523-4abc-abd7-157dae79bd7d","Type":"ContainerStarted","Data":"daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6"}
Jul 17 06:25:24.489130 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:24.489041    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99" event={"ID":"d8c527b3-f523-4abc-abd7-157dae79bd7d","Type":"ContainerStarted","Data":"fb7b1712818a108a21aa5c80c4cfa07f746904dda0cba1a03ace0ee89845a484"}
Jul 17 06:25:24.522976 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:24.522908    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99" podStartSLOduration=1.5228859959999999 podStartE2EDuration="1.522885996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:25:24.522660248 +0000 UTC m=+3149.301143205" watchObservedRunningTime="2026-07-17 06:25:24.522885996 +0000 UTC m=+3149.301368952"
Jul 17 06:25:25.495876 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:25.495831    2559 generic.go:356] "Generic (PLEG): container finished" podID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerID="984c1917afa7ccd0cf2775b9fd02f58f297aa919f19ec83b541a5a0d764a2cdf" exitCode=6
Jul 17 06:25:25.496356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:25.495914    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" event={"ID":"d7f96911-334e-49d7-bd92-258ca8b5c2a8","Type":"ContainerDied","Data":"984c1917afa7ccd0cf2775b9fd02f58f297aa919f19ec83b541a5a0d764a2cdf"}
Jul 17 06:25:25.496356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:25.495971    2559 scope.go:117] "RemoveContainer" containerID="4ffaf6ec58a7ee60e29b6b2b9c5ed2ed06c85e00f44be4db127be1081c5fdbbd"
Jul 17 06:25:25.496356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:25.496238    2559 scope.go:117] "RemoveContainer" containerID="984c1917afa7ccd0cf2775b9fd02f58f297aa919f19ec83b541a5a0d764a2cdf"
Jul 17 06:25:25.496533 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:25.496469    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx_odh-ai-gateway-infra(d7f96911-334e-49d7-bd92-258ca8b5c2a8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8"
Jul 17 06:25:26.502346 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:26.502315    2559 scope.go:117] "RemoveContainer" containerID="984c1917afa7ccd0cf2775b9fd02f58f297aa919f19ec83b541a5a0d764a2cdf"
Jul 17 06:25:26.502748 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:26.502555    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx_odh-ai-gateway-infra(d7f96911-334e-49d7-bd92-258ca8b5c2a8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8"
Jul 17 06:25:33.544051 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:33.544006    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:33.544051 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:33.544066    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:33.556845 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:33.556812    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:34.554246 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:34.554208    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:25:36.747083 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:36.747049    2559 scope.go:117] "RemoveContainer" containerID="984c1917afa7ccd0cf2775b9fd02f58f297aa919f19ec83b541a5a0d764a2cdf"
Jul 17 06:25:36.747494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:36.747155    2559 scope.go:117] "RemoveContainer" containerID="ce6aaddb4834bf5a0572fb01c4baa4712be15c576a26ef6efbe4f746f1e418ae"
Jul 17 06:25:37.556827 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.556791    2559 generic.go:356] "Generic (PLEG): container finished" podID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerID="440d3fcf5fb7133612fcddb828a25e3e7bdf0efd7d4b4b5c58c2f33118b14e3f" exitCode=6
Jul 17 06:25:37.557046 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.556878    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" event={"ID":"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4","Type":"ContainerDied","Data":"440d3fcf5fb7133612fcddb828a25e3e7bdf0efd7d4b4b5c58c2f33118b14e3f"}
Jul 17 06:25:37.557046 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.556939    2559 scope.go:117] "RemoveContainer" containerID="ce6aaddb4834bf5a0572fb01c4baa4712be15c576a26ef6efbe4f746f1e418ae"
Jul 17 06:25:37.557226 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.557204    2559 scope.go:117] "RemoveContainer" containerID="440d3fcf5fb7133612fcddb828a25e3e7bdf0efd7d4b4b5c58c2f33118b14e3f"
Jul 17 06:25:37.557505 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:37.557483    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw_odh-ai-gateway-infra(5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4"
Jul 17 06:25:37.558986 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.558966    2559 generic.go:356] "Generic (PLEG): container finished" podID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerID="803f5438556b946f41cfa04411246857f7cde906f1c3cbf696effd3f295aee7c" exitCode=6
Jul 17 06:25:37.559055 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.559027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" event={"ID":"d7f96911-334e-49d7-bd92-258ca8b5c2a8","Type":"ContainerDied","Data":"803f5438556b946f41cfa04411246857f7cde906f1c3cbf696effd3f295aee7c"}
Jul 17 06:25:37.559288 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.559276    2559 scope.go:117] "RemoveContainer" containerID="803f5438556b946f41cfa04411246857f7cde906f1c3cbf696effd3f295aee7c"
Jul 17 06:25:37.559493 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:37.559477    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx_odh-ai-gateway-infra(d7f96911-334e-49d7-bd92-258ca8b5c2a8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8"
Jul 17 06:25:37.569828 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:37.569803    2559 scope.go:117] "RemoveContainer" containerID="984c1917afa7ccd0cf2775b9fd02f58f297aa919f19ec83b541a5a0d764a2cdf"
Jul 17 06:25:38.584978 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.584934    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"]
Jul 17 06:25:38.603323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.603288    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"]
Jul 17 06:25:38.738332 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.738225    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:38.775526 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.775497    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:38.909648 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.909549    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f96911-334e-49d7-bd92-258ca8b5c2a8-maas-api-service-ca\") pod \"d7f96911-334e-49d7-bd92-258ca8b5c2a8\" (UID: \"d7f96911-334e-49d7-bd92-258ca8b5c2a8\") "
Jul 17 06:25:38.909648 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.909645    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4-maas-api-service-ca\") pod \"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4\" (UID: \"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4\") "
Jul 17 06:25:38.910059 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.910031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7f96911-334e-49d7-bd92-258ca8b5c2a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7f96911-334e-49d7-bd92-258ca8b5c2a8" (UID: "d7f96911-334e-49d7-bd92-258ca8b5c2a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:25:38.910139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.910070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" (UID: "5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:25:38.910650 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.910620    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f96911-334e-49d7-bd92-258ca8b5c2a8-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:38.910755 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:38.910653    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:25:39.580305 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.580265    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"
Jul 17 06:25:39.580305 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.580291    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw" event={"ID":"5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4","Type":"ContainerDied","Data":"8f6d4c87dfc4f7b32afa69ed679a5fe59fea643bd5c168e6aec7e425bbec8136"}
Jul 17 06:25:39.580579 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.580392    2559 scope.go:117] "RemoveContainer" containerID="440d3fcf5fb7133612fcddb828a25e3e7bdf0efd7d4b4b5c58c2f33118b14e3f"
Jul 17 06:25:39.581736 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.581715    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"
Jul 17 06:25:39.581878 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.581708    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx" event={"ID":"d7f96911-334e-49d7-bd92-258ca8b5c2a8","Type":"ContainerDied","Data":"e444d56efe572bd4a8f06a6f2633ce411488eedb15664fc75a3e67d858bdec41"}
Jul 17 06:25:39.590508 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.590333    2559 scope.go:117] "RemoveContainer" containerID="803f5438556b946f41cfa04411246857f7cde906f1c3cbf696effd3f295aee7c"
Jul 17 06:25:39.605028 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.604996    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"]
Jul 17 06:25:39.612786 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.612754    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-jc4mw"]
Jul 17 06:25:39.626292 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.626245    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"]
Jul 17 06:25:39.628347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.628317    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-f5qfx"]
Jul 17 06:25:39.752408 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.752368    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" path="/var/lib/kubelet/pods/5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4/volumes"
Jul 17 06:25:39.752743 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:39.752730    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" path="/var/lib/kubelet/pods/d7f96911-334e-49d7-bd92-258ca8b5c2a8/volumes"
Jul 17 06:25:51.300680 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.300642    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"]
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301032    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301046    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301062    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301067    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301073    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301079    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301090    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301097    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301109    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301117    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301171    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301179    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301185    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301192    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f96911-334e-49d7-bd92-258ca8b5c2a8" containerName="revoke-keys"
Jul 17 06:25:51.301211 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.301199    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:25:51.305995 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.305974    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:25:51.310084 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.310054    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:25:51.310390 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.310053    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:25:51.311491 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.311465    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"]
Jul 17 06:25:51.407176 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.407128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e67da3a4-45dc-4183-89bb-41a9dde53204-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-bfpl2\" (UID: \"e67da3a4-45dc-4183-89bb-41a9dde53204\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:25:51.508039 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.507999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e67da3a4-45dc-4183-89bb-41a9dde53204-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-bfpl2\" (UID: \"e67da3a4-45dc-4183-89bb-41a9dde53204\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:25:51.508685 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.508655    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e67da3a4-45dc-4183-89bb-41a9dde53204-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-bfpl2\" (UID: \"e67da3a4-45dc-4183-89bb-41a9dde53204\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:25:51.618748 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.618649    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:25:51.762282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:51.762247    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"]
Jul 17 06:25:51.767599 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:25:51.767569    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67da3a4_45dc_4183_89bb_41a9dde53204.slice/crio-ec766a82b3d375f56895d56bad7ee4b9e042a364b0a475acd4e5e2b59973e028 WatchSource:0}: Error finding container ec766a82b3d375f56895d56bad7ee4b9e042a364b0a475acd4e5e2b59973e028: Status 404 returned error can't find the container with id ec766a82b3d375f56895d56bad7ee4b9e042a364b0a475acd4e5e2b59973e028
Jul 17 06:25:52.652188 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:52.652092    2559 generic.go:356] "Generic (PLEG): container finished" podID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerID="65ba533f4acfd90546b0110d2171a3b0eb57a8ae99287b2950d852c93fcc291d" exitCode=6
Jul 17 06:25:52.652188 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:52.652158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" event={"ID":"e67da3a4-45dc-4183-89bb-41a9dde53204","Type":"ContainerDied","Data":"65ba533f4acfd90546b0110d2171a3b0eb57a8ae99287b2950d852c93fcc291d"}
Jul 17 06:25:52.652601 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:52.652210    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" event={"ID":"e67da3a4-45dc-4183-89bb-41a9dde53204","Type":"ContainerStarted","Data":"ec766a82b3d375f56895d56bad7ee4b9e042a364b0a475acd4e5e2b59973e028"}
Jul 17 06:25:52.652601 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:52.652415    2559 scope.go:117] "RemoveContainer" containerID="65ba533f4acfd90546b0110d2171a3b0eb57a8ae99287b2950d852c93fcc291d"
Jul 17 06:25:53.659079 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:53.658991    2559 generic.go:356] "Generic (PLEG): container finished" podID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerID="dfefee3de60ca586d9ea66b45530f7901c05b694961dec680017265706614ea3" exitCode=6
Jul 17 06:25:53.659493 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:53.659070    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" event={"ID":"e67da3a4-45dc-4183-89bb-41a9dde53204","Type":"ContainerDied","Data":"dfefee3de60ca586d9ea66b45530f7901c05b694961dec680017265706614ea3"}
Jul 17 06:25:53.659493 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:53.659141    2559 scope.go:117] "RemoveContainer" containerID="65ba533f4acfd90546b0110d2171a3b0eb57a8ae99287b2950d852c93fcc291d"
Jul 17 06:25:53.659493 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:53.659416    2559 scope.go:117] "RemoveContainer" containerID="dfefee3de60ca586d9ea66b45530f7901c05b694961dec680017265706614ea3"
Jul 17 06:25:53.659638 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:53.659620    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-bfpl2_odh-ai-gateway-infra(e67da3a4-45dc-4183-89bb-41a9dde53204)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204"
Jul 17 06:25:54.131780 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:54.131748    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:25:54.666931 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:25:54.666902    2559 scope.go:117] "RemoveContainer" containerID="dfefee3de60ca586d9ea66b45530f7901c05b694961dec680017265706614ea3"
Jul 17 06:25:54.667420 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:25:54.667181    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-bfpl2_odh-ai-gateway-infra(e67da3a4-45dc-4183-89bb-41a9dde53204)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204"
Jul 17 06:26:07.889966 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:07.889932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:07.954773 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:07.954733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:09.746409 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:09.746377    2559 scope.go:117] "RemoveContainer" containerID="dfefee3de60ca586d9ea66b45530f7901c05b694961dec680017265706614ea3"
Jul 17 06:26:10.746081 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:10.746045    2559 generic.go:356] "Generic (PLEG): container finished" podID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerID="2703fb5c1f2db7f88d31b464d38e9beb3b5f6fb4802ff246699af692b8894f60" exitCode=6
Jul 17 06:26:10.746258 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:10.746112    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" event={"ID":"e67da3a4-45dc-4183-89bb-41a9dde53204","Type":"ContainerDied","Data":"2703fb5c1f2db7f88d31b464d38e9beb3b5f6fb4802ff246699af692b8894f60"}
Jul 17 06:26:10.746258 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:10.746172    2559 scope.go:117] "RemoveContainer" containerID="dfefee3de60ca586d9ea66b45530f7901c05b694961dec680017265706614ea3"
Jul 17 06:26:10.746497 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:10.746481    2559 scope.go:117] "RemoveContainer" containerID="2703fb5c1f2db7f88d31b464d38e9beb3b5f6fb4802ff246699af692b8894f60"
Jul 17 06:26:10.746821 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:26:10.746679    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-bfpl2_odh-ai-gateway-infra(e67da3a4-45dc-4183-89bb-41a9dde53204)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204"
Jul 17 06:26:11.774122 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:11.774081    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"]
Jul 17 06:26:11.915703 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:11.915673    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:26:12.001345 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.001306    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e67da3a4-45dc-4183-89bb-41a9dde53204-maas-api-service-ca\") pod \"e67da3a4-45dc-4183-89bb-41a9dde53204\" (UID: \"e67da3a4-45dc-4183-89bb-41a9dde53204\") "
Jul 17 06:26:12.001741 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.001708    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e67da3a4-45dc-4183-89bb-41a9dde53204-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e67da3a4-45dc-4183-89bb-41a9dde53204" (UID: "e67da3a4-45dc-4183-89bb-41a9dde53204"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:26:12.103008 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.102918    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e67da3a4-45dc-4183-89bb-41a9dde53204-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:26:12.758713 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.758678    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2" event={"ID":"e67da3a4-45dc-4183-89bb-41a9dde53204","Type":"ContainerDied","Data":"ec766a82b3d375f56895d56bad7ee4b9e042a364b0a475acd4e5e2b59973e028"}
Jul 17 06:26:12.758935 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.758742    2559 scope.go:117] "RemoveContainer" containerID="2703fb5c1f2db7f88d31b464d38e9beb3b5f6fb4802ff246699af692b8894f60"
Jul 17 06:26:12.758935 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.758687    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"
Jul 17 06:26:12.782771 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.782730    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"]
Jul 17 06:26:12.790350 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:12.790317    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-bfpl2"]
Jul 17 06:26:13.752250 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:13.752210    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" path="/var/lib/kubelet/pods/e67da3a4-45dc-4183-89bb-41a9dde53204/volumes"
Jul 17 06:26:18.656380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656340    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"]
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656705    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656717    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656734    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656739    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656754    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656760    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656812    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656821    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656826    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:18.656988 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.656833    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3adc9e-8bf9-4dd4-8282-04ea81c21ed4" containerName="revoke-keys"
Jul 17 06:26:18.661208 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.661189    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:18.664946 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.664673    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:26:18.664946 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.664790    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:26:18.675747 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.675547    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"]
Jul 17 06:26:18.767283 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.767238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/080c9a03-9884-420b-b6f3-663eb87cbfa2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn\" (UID: \"080c9a03-9884-420b-b6f3-663eb87cbfa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:18.868001 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.867958    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/080c9a03-9884-420b-b6f3-663eb87cbfa2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn\" (UID: \"080c9a03-9884-420b-b6f3-663eb87cbfa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:18.868829 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.868804    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/080c9a03-9884-420b-b6f3-663eb87cbfa2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn\" (UID: \"080c9a03-9884-420b-b6f3-663eb87cbfa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:18.980556 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:18.980514    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:19.128140 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:19.128106    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"]
Jul 17 06:26:19.129199 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:26:19.129174    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080c9a03_9884_420b_b6f3_663eb87cbfa2.slice/crio-09c8bc9d3b83f9259957ce99db06b512ecc74bc27eee80d4953f257f07e490da WatchSource:0}: Error finding container 09c8bc9d3b83f9259957ce99db06b512ecc74bc27eee80d4953f257f07e490da: Status 404 returned error can't find the container with id 09c8bc9d3b83f9259957ce99db06b512ecc74bc27eee80d4953f257f07e490da
Jul 17 06:26:19.792698 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:19.792667    2559 generic.go:356] "Generic (PLEG): container finished" podID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerID="d4e4647f5be2d2dc6fe3943ef67da89e5960094534939295908855a40cffcd25" exitCode=6
Jul 17 06:26:19.793108 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:19.792749    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" event={"ID":"080c9a03-9884-420b-b6f3-663eb87cbfa2","Type":"ContainerDied","Data":"d4e4647f5be2d2dc6fe3943ef67da89e5960094534939295908855a40cffcd25"}
Jul 17 06:26:19.793108 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:19.792812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" event={"ID":"080c9a03-9884-420b-b6f3-663eb87cbfa2","Type":"ContainerStarted","Data":"09c8bc9d3b83f9259957ce99db06b512ecc74bc27eee80d4953f257f07e490da"}
Jul 17 06:26:19.793108 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:19.792952    2559 scope.go:117] "RemoveContainer" containerID="d4e4647f5be2d2dc6fe3943ef67da89e5960094534939295908855a40cffcd25"
Jul 17 06:26:20.799615 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:20.799578    2559 generic.go:356] "Generic (PLEG): container finished" podID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerID="1c33b405c33cb029cc4eea923edce3e1a8a57e2d03c5f7aa2fe967ea2d7e62d5" exitCode=6
Jul 17 06:26:20.800072 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:20.799631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" event={"ID":"080c9a03-9884-420b-b6f3-663eb87cbfa2","Type":"ContainerDied","Data":"1c33b405c33cb029cc4eea923edce3e1a8a57e2d03c5f7aa2fe967ea2d7e62d5"}
Jul 17 06:26:20.800072 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:20.799686    2559 scope.go:117] "RemoveContainer" containerID="d4e4647f5be2d2dc6fe3943ef67da89e5960094534939295908855a40cffcd25"
Jul 17 06:26:20.800072 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:20.799899    2559 scope.go:117] "RemoveContainer" containerID="1c33b405c33cb029cc4eea923edce3e1a8a57e2d03c5f7aa2fe967ea2d7e62d5"
Jul 17 06:26:20.800193 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:26:20.800158    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn_odh-ai-gateway-infra(080c9a03-9884-420b-b6f3-663eb87cbfa2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2"
Jul 17 06:26:21.806581 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:21.806551    2559 scope.go:117] "RemoveContainer" containerID="1c33b405c33cb029cc4eea923edce3e1a8a57e2d03c5f7aa2fe967ea2d7e62d5"
Jul 17 06:26:21.807026 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:26:21.806772    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn_odh-ai-gateway-infra(080c9a03-9884-420b-b6f3-663eb87cbfa2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2"
Jul 17 06:26:35.678052 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:35.678001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:35.748576 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:35.748549    2559 scope.go:117] "RemoveContainer" containerID="1c33b405c33cb029cc4eea923edce3e1a8a57e2d03c5f7aa2fe967ea2d7e62d5"
Jul 17 06:26:35.778127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:35.778093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:36.883873 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:36.883831    2559 generic.go:356] "Generic (PLEG): container finished" podID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerID="36bc11b1d36c1a106fdac57399eed6354ccd856e62a09aa16ca273880b0623cf" exitCode=6
Jul 17 06:26:36.884321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:36.883909    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" event={"ID":"080c9a03-9884-420b-b6f3-663eb87cbfa2","Type":"ContainerDied","Data":"36bc11b1d36c1a106fdac57399eed6354ccd856e62a09aa16ca273880b0623cf"}
Jul 17 06:26:36.884321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:36.883975    2559 scope.go:117] "RemoveContainer" containerID="1c33b405c33cb029cc4eea923edce3e1a8a57e2d03c5f7aa2fe967ea2d7e62d5"
Jul 17 06:26:36.884321 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:36.884319    2559 scope.go:117] "RemoveContainer" containerID="36bc11b1d36c1a106fdac57399eed6354ccd856e62a09aa16ca273880b0623cf"
Jul 17 06:26:36.884548 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:26:36.884530    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn_odh-ai-gateway-infra(080c9a03-9884-420b-b6f3-663eb87cbfa2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2"
Jul 17 06:26:37.910167 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:37.910126    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"]
Jul 17 06:26:38.048100 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.048064    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:38.145127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.145091    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/080c9a03-9884-420b-b6f3-663eb87cbfa2-maas-api-service-ca\") pod \"080c9a03-9884-420b-b6f3-663eb87cbfa2\" (UID: \"080c9a03-9884-420b-b6f3-663eb87cbfa2\") "
Jul 17 06:26:38.145472 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.145446    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/080c9a03-9884-420b-b6f3-663eb87cbfa2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "080c9a03-9884-420b-b6f3-663eb87cbfa2" (UID: "080c9a03-9884-420b-b6f3-663eb87cbfa2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:26:38.246065 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.246024    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/080c9a03-9884-420b-b6f3-663eb87cbfa2-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:26:38.897061 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.897029    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"
Jul 17 06:26:38.897061 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.897039    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn" event={"ID":"080c9a03-9884-420b-b6f3-663eb87cbfa2","Type":"ContainerDied","Data":"09c8bc9d3b83f9259957ce99db06b512ecc74bc27eee80d4953f257f07e490da"}
Jul 17 06:26:38.897347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.897096    2559 scope.go:117] "RemoveContainer" containerID="36bc11b1d36c1a106fdac57399eed6354ccd856e62a09aa16ca273880b0623cf"
Jul 17 06:26:38.922311 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.922274    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"]
Jul 17 06:26:38.927780 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:38.927750    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-qgqwn"]
Jul 17 06:26:39.752574 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:39.752529    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" path="/var/lib/kubelet/pods/080c9a03-9884-420b-b6f3-663eb87cbfa2/volumes"
Jul 17 06:26:46.700007 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:46.699970    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"]
Jul 17 06:26:46.722746 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:46.722707    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:47.474518 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:47.474437    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:51.240495 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240461    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"]
Jul 17 06:26:51.240923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240814    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:26:51.240923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240825    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:26:51.240923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240833    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:26:51.240923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240839    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:26:51.240923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240849    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:51.240923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240868    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67da3a4-45dc-4183-89bb-41a9dde53204" containerName="revoke-keys"
Jul 17 06:26:51.241152 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240944    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:26:51.241152 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.240954    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:26:51.244040 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.244023    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.246831 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.246810    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e9bcaa\"/\"rate-test-model-e9bcaa-kserve-self-signed-certs\""
Jul 17 06:26:51.253887 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.253842    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"]
Jul 17 06:26:51.329259 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.329226    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:26:51.363637 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.363588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-dshm\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.363637 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.363642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-model-cache\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.364005 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.363756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-tmp-dir\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.364005 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.363797    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pw7j\" (UniqueName: \"kubernetes.io/projected/cc74b36f-4364-49e2-9489-f74de93d7d20-kube-api-access-2pw7j\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.364005 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.363935    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-home\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.364180 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.364011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc74b36f-4364-49e2-9489-f74de93d7d20-tls-certs\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465042 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.464998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-dshm\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465042 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-model-cache\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-tmp-dir\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pw7j\" (UniqueName: \"kubernetes.io/projected/cc74b36f-4364-49e2-9489-f74de93d7d20-kube-api-access-2pw7j\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-home\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc74b36f-4364-49e2-9489-f74de93d7d20-tls-certs\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465582 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465536    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-tmp-dir\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465715 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-home\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.465797 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.465732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-model-cache\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.467392 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.467360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-dshm\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.468165 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.468144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc74b36f-4364-49e2-9489-f74de93d7d20-tls-certs\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.473533 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.473508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pw7j\" (UniqueName: \"kubernetes.io/projected/cc74b36f-4364-49e2-9489-f74de93d7d20-kube-api-access-2pw7j\") pod \"rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") " pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.554543 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.554447    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:26:51.699828 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.699743    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"]
Jul 17 06:26:51.702778 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:26:51.702750    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc74b36f_4364_49e2_9489_f74de93d7d20.slice/crio-35d279fd0a550f5c30018791e3303d8ec08994ef0ceb263876d3d22cb8fd2afd WatchSource:0}: Error finding container 35d279fd0a550f5c30018791e3303d8ec08994ef0ceb263876d3d22cb8fd2afd: Status 404 returned error can't find the container with id 35d279fd0a550f5c30018791e3303d8ec08994ef0ceb263876d3d22cb8fd2afd
Jul 17 06:26:51.960142 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.960044    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz" event={"ID":"cc74b36f-4364-49e2-9489-f74de93d7d20","Type":"ContainerStarted","Data":"ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312"}
Jul 17 06:26:51.960142 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.960095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz" event={"ID":"cc74b36f-4364-49e2-9489-f74de93d7d20","Type":"ContainerStarted","Data":"35d279fd0a550f5c30018791e3303d8ec08994ef0ceb263876d3d22cb8fd2afd"}
Jul 17 06:26:51.980666 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:26:51.980609    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz" podStartSLOduration=0.980591588 podStartE2EDuration="980.591588ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:26:51.979916843 +0000 UTC m=+3236.758399804" watchObservedRunningTime="2026-07-17 06:26:51.980591588 +0000 UTC m=+3236.759074547"
Jul 17 06:27:01.554853 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:01.554809    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:27:01.555400 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:01.554902    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:27:01.568430 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:01.568398    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:27:01.720029 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:01.719985    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99" podUID="d8c527b3-f523-4abc-abd7-157dae79bd7d" containerName="main" containerID="cri-o://daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6" gracePeriod=45
Jul 17 06:27:01.987196 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:01.987165    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:27:02.014642 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.014610    2559 generic.go:356] "Generic (PLEG): container finished" podID="d8c527b3-f523-4abc-abd7-157dae79bd7d" containerID="daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6" exitCode=0
Jul 17 06:27:02.014841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.014659    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"
Jul 17 06:27:02.014841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.014700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99" event={"ID":"d8c527b3-f523-4abc-abd7-157dae79bd7d","Type":"ContainerDied","Data":"daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6"}
Jul 17 06:27:02.014841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.014742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99" event={"ID":"d8c527b3-f523-4abc-abd7-157dae79bd7d","Type":"ContainerDied","Data":"fb7b1712818a108a21aa5c80c4cfa07f746904dda0cba1a03ace0ee89845a484"}
Jul 17 06:27:02.014841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.014764    2559 scope.go:117] "RemoveContainer" containerID="daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6"
Jul 17 06:27:02.024948 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.024925    2559 scope.go:117] "RemoveContainer" containerID="daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6"
Jul 17 06:27:02.025302 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:27:02.025280    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6\": container with ID starting with daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6 not found: ID does not exist" containerID="daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6"
Jul 17 06:27:02.025377 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.025317    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6"} err="failed to get container status \"daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6\": rpc error: code = NotFound desc = could not find container \"daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6\": container with ID starting with daffa7477ee2f5ea59acbba06a00af5de8355374e46396e998cbf9b22576d8b6 not found: ID does not exist"
Jul 17 06:27:02.028479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.028454    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:27:02.067706 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067658    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-model-cache\") pod \"d8c527b3-f523-4abc-abd7-157dae79bd7d\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") "
Jul 17 06:27:02.067928 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067724    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-home\") pod \"d8c527b3-f523-4abc-abd7-157dae79bd7d\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") "
Jul 17 06:27:02.067928 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067757    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-tmp-dir\") pod \"d8c527b3-f523-4abc-abd7-157dae79bd7d\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") "
Jul 17 06:27:02.067928 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067790    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9hlx\" (UniqueName: \"kubernetes.io/projected/d8c527b3-f523-4abc-abd7-157dae79bd7d-kube-api-access-f9hlx\") pod \"d8c527b3-f523-4abc-abd7-157dae79bd7d\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") "
Jul 17 06:27:02.067928 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067844    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8c527b3-f523-4abc-abd7-157dae79bd7d-tls-certs\") pod \"d8c527b3-f523-4abc-abd7-157dae79bd7d\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") "
Jul 17 06:27:02.068162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067944    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-dshm\") pod \"d8c527b3-f523-4abc-abd7-157dae79bd7d\" (UID: \"d8c527b3-f523-4abc-abd7-157dae79bd7d\") "
Jul 17 06:27:02.068162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067955    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d8c527b3-f523-4abc-abd7-157dae79bd7d" (UID: "d8c527b3-f523-4abc-abd7-157dae79bd7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:27:02.068162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.067987    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-home" (OuterVolumeSpecName: "home") pod "d8c527b3-f523-4abc-abd7-157dae79bd7d" (UID: "d8c527b3-f523-4abc-abd7-157dae79bd7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:27:02.068162 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.068036    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d8c527b3-f523-4abc-abd7-157dae79bd7d" (UID: "d8c527b3-f523-4abc-abd7-157dae79bd7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:27:02.068367 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.068270    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:02.068367 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.068289    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:02.068367 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.068302    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:02.070392 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.070360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-dshm" (OuterVolumeSpecName: "dshm") pod "d8c527b3-f523-4abc-abd7-157dae79bd7d" (UID: "d8c527b3-f523-4abc-abd7-157dae79bd7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:27:02.070654 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.070629    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c527b3-f523-4abc-abd7-157dae79bd7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d8c527b3-f523-4abc-abd7-157dae79bd7d" (UID: "d8c527b3-f523-4abc-abd7-157dae79bd7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:27:02.070833 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.070813    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c527b3-f523-4abc-abd7-157dae79bd7d-kube-api-access-f9hlx" (OuterVolumeSpecName: "kube-api-access-f9hlx") pod "d8c527b3-f523-4abc-abd7-157dae79bd7d" (UID: "d8c527b3-f523-4abc-abd7-157dae79bd7d"). InnerVolumeSpecName "kube-api-access-f9hlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:27:02.169841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.169746    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9hlx\" (UniqueName: \"kubernetes.io/projected/d8c527b3-f523-4abc-abd7-157dae79bd7d-kube-api-access-f9hlx\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:02.169841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.169781    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8c527b3-f523-4abc-abd7-157dae79bd7d-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:02.169841 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.169792    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8c527b3-f523-4abc-abd7-157dae79bd7d-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:02.336154 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.336114    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"]
Jul 17 06:27:02.340725 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:02.340694    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/sub-test-model-e9bcaa-kserve-5cdf9fc47c-rpv99"]
Jul 17 06:27:03.030977 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.030946    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"]
Jul 17 06:27:03.031384 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.031292    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:27:03.031384 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.031303    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:27:03.031384 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.031313    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8c527b3-f523-4abc-abd7-157dae79bd7d" containerName="main"
Jul 17 06:27:03.031384 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.031318    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8c527b3-f523-4abc-abd7-157dae79bd7d" containerName="main"
Jul 17 06:27:03.031384 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.031376    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="080c9a03-9884-420b-b6f3-663eb87cbfa2" containerName="revoke-keys"
Jul 17 06:27:03.031384 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.031385    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8c527b3-f523-4abc-abd7-157dae79bd7d" containerName="main"
Jul 17 06:27:03.036058 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.036038    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:03.039053 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.039034    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:27:03.039053 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.039047    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:27:03.044305 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.044274    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"]
Jul 17 06:27:03.180616 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.180575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0042bba9-a026-4116-9bc4-246f80c7f4d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vlflw\" (UID: \"0042bba9-a026-4116-9bc4-246f80c7f4d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:03.281899 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.281775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0042bba9-a026-4116-9bc4-246f80c7f4d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vlflw\" (UID: \"0042bba9-a026-4116-9bc4-246f80c7f4d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:03.282451 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.282431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0042bba9-a026-4116-9bc4-246f80c7f4d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-vlflw\" (UID: \"0042bba9-a026-4116-9bc4-246f80c7f4d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:03.347435 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.347389    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:03.488399 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.488373    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"]
Jul 17 06:27:03.491195 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:27:03.491156    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0042bba9_a026_4116_9bc4_246f80c7f4d2.slice/crio-75018d2a2afc4651b96e18c1c1659ce132ba79c0b2b0fde283924b939d4681d1 WatchSource:0}: Error finding container 75018d2a2afc4651b96e18c1c1659ce132ba79c0b2b0fde283924b939d4681d1: Status 404 returned error can't find the container with id 75018d2a2afc4651b96e18c1c1659ce132ba79c0b2b0fde283924b939d4681d1
Jul 17 06:27:03.758674 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:03.758635    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8c527b3-f523-4abc-abd7-157dae79bd7d" path="/var/lib/kubelet/pods/d8c527b3-f523-4abc-abd7-157dae79bd7d/volumes"
Jul 17 06:27:04.027023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:04.026984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerStarted","Data":"7da3e6a04b6d1d59515f47f07bf262b160ab91ef81aa1a8bbeffc90f62df3c8d"}
Jul 17 06:27:04.027249 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:04.027046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerStarted","Data":"75018d2a2afc4651b96e18c1c1659ce132ba79c0b2b0fde283924b939d4681d1"}
Jul 17 06:27:04.049236 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:04.049146    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" podStartSLOduration=1.049124621 podStartE2EDuration="1.049124621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:27:04.04897171 +0000 UTC m=+3248.827454671" watchObservedRunningTime="2026-07-17 06:27:04.049124621 +0000 UTC m=+3248.827607584"
Jul 17 06:27:05.033127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:05.033091    2559 generic.go:356] "Generic (PLEG): container finished" podID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerID="7da3e6a04b6d1d59515f47f07bf262b160ab91ef81aa1a8bbeffc90f62df3c8d" exitCode=6
Jul 17 06:27:05.033379 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:05.033175    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerDied","Data":"7da3e6a04b6d1d59515f47f07bf262b160ab91ef81aa1a8bbeffc90f62df3c8d"}
Jul 17 06:27:05.033454 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:05.033440    2559 scope.go:117] "RemoveContainer" containerID="7da3e6a04b6d1d59515f47f07bf262b160ab91ef81aa1a8bbeffc90f62df3c8d"
Jul 17 06:27:06.040573 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:06.040541    2559 generic.go:356] "Generic (PLEG): container finished" podID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerID="04e8986da014ce5308a95c7064d3a0afdc97d973734077303ca1efab48a62e80" exitCode=6
Jul 17 06:27:06.041026 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:06.040624    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerDied","Data":"04e8986da014ce5308a95c7064d3a0afdc97d973734077303ca1efab48a62e80"}
Jul 17 06:27:06.041026 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:06.040691    2559 scope.go:117] "RemoveContainer" containerID="7da3e6a04b6d1d59515f47f07bf262b160ab91ef81aa1a8bbeffc90f62df3c8d"
Jul 17 06:27:06.041026 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:06.040914    2559 scope.go:117] "RemoveContainer" containerID="04e8986da014ce5308a95c7064d3a0afdc97d973734077303ca1efab48a62e80"
Jul 17 06:27:06.041177 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:27:06.041150    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vlflw_odh-ai-gateway-infra(0042bba9-a026-4116-9bc4-246f80c7f4d2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2"
Jul 17 06:27:07.047672 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:07.047643    2559 scope.go:117] "RemoveContainer" containerID="04e8986da014ce5308a95c7064d3a0afdc97d973734077303ca1efab48a62e80"
Jul 17 06:27:07.048112 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:27:07.047906    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vlflw_odh-ai-gateway-infra(0042bba9-a026-4116-9bc4-246f80c7f4d2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2"
Jul 17 06:27:09.146693 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:09.146660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:09.247473 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:09.247431    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:11.853690 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:11.853657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:11.952215 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:11.952182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:17.682811 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:17.682774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:17.785285 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:17.785252    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:17.891739 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:17.891705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:21.749045 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:21.749013    2559 scope.go:117] "RemoveContainer" containerID="04e8986da014ce5308a95c7064d3a0afdc97d973734077303ca1efab48a62e80"
Jul 17 06:27:22.120716 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:22.120679    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerStarted","Data":"ee6f72a028ee9643276936613fe115aaa188da54e0ff20f804de31176b16e3ce"}
Jul 17 06:27:23.127453 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:23.127416    2559 generic.go:356] "Generic (PLEG): container finished" podID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerID="ee6f72a028ee9643276936613fe115aaa188da54e0ff20f804de31176b16e3ce" exitCode=6
Jul 17 06:27:23.127940 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:23.127488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerDied","Data":"ee6f72a028ee9643276936613fe115aaa188da54e0ff20f804de31176b16e3ce"}
Jul 17 06:27:23.127940 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:23.127551    2559 scope.go:117] "RemoveContainer" containerID="04e8986da014ce5308a95c7064d3a0afdc97d973734077303ca1efab48a62e80"
Jul 17 06:27:23.127940 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:23.127839    2559 scope.go:117] "RemoveContainer" containerID="ee6f72a028ee9643276936613fe115aaa188da54e0ff20f804de31176b16e3ce"
Jul 17 06:27:23.128151 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:27:23.128107    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-vlflw_odh-ai-gateway-infra(0042bba9-a026-4116-9bc4-246f80c7f4d2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2"
Jul 17 06:27:23.182547 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:23.182505    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"]
Jul 17 06:27:24.273966 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:24.273933    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:24.284109 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:24.284072    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0042bba9-a026-4116-9bc4-246f80c7f4d2-maas-api-service-ca\") pod \"0042bba9-a026-4116-9bc4-246f80c7f4d2\" (UID: \"0042bba9-a026-4116-9bc4-246f80c7f4d2\") "
Jul 17 06:27:24.284480 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:24.284453    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0042bba9-a026-4116-9bc4-246f80c7f4d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0042bba9-a026-4116-9bc4-246f80c7f4d2" (UID: "0042bba9-a026-4116-9bc4-246f80c7f4d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:27:24.384772 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:24.384668    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0042bba9-a026-4116-9bc4-246f80c7f4d2-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:27:25.140105 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:25.140054    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw" event={"ID":"0042bba9-a026-4116-9bc4-246f80c7f4d2","Type":"ContainerDied","Data":"75018d2a2afc4651b96e18c1c1659ce132ba79c0b2b0fde283924b939d4681d1"}
Jul 17 06:27:25.140323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:25.140132    2559 scope.go:117] "RemoveContainer" containerID="ee6f72a028ee9643276936613fe115aaa188da54e0ff20f804de31176b16e3ce"
Jul 17 06:27:25.140323 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:25.140068    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"
Jul 17 06:27:25.161940 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:25.161908    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"]
Jul 17 06:27:25.166166 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:25.166138    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-vlflw"]
Jul 17 06:27:25.753425 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:25.753393    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" path="/var/lib/kubelet/pods/0042bba9-a026-4116-9bc4-246f80c7f4d2/volumes"
Jul 17 06:27:30.167326 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:30.167291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:27:44.707063 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707026    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"]
Jul 17 06:27:44.707470 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707422    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.707470 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707435    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.707470 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707461    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.707470 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707466    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.707604 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707522    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.707604 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707529    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.707604 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.707538    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:27:44.710540 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.710523    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:27:44.713118 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.713095    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:27:44.713231 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.713098    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:27:44.717315 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.717286    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"]
Jul 17 06:27:44.772256 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.772218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-stwks\" (UID: \"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:27:44.873496 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.873461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-stwks\" (UID: \"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:27:44.874191 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:44.874162    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-stwks\" (UID: \"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:27:45.022899 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:45.022846    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:27:45.160298 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:45.160272    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"]
Jul 17 06:27:45.163187 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:27:45.163154    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d0c8d7_6ae0_48ee_8f12_ed8789ccffc4.slice/crio-26f2b87c0d84cd7f8bfbec05818495818a722a27ea59c03bdd91edb040aafde9 WatchSource:0}: Error finding container 26f2b87c0d84cd7f8bfbec05818495818a722a27ea59c03bdd91edb040aafde9: Status 404 returned error can't find the container with id 26f2b87c0d84cd7f8bfbec05818495818a722a27ea59c03bdd91edb040aafde9
Jul 17 06:27:45.252961 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:45.252924    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" event={"ID":"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4","Type":"ContainerStarted","Data":"26f2b87c0d84cd7f8bfbec05818495818a722a27ea59c03bdd91edb040aafde9"}
Jul 17 06:27:46.259510 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:46.259475    2559 generic.go:356] "Generic (PLEG): container finished" podID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerID="3f954c446a86d706757d46e23c6c72f2db6322928513caea7f9b5940d80ead45" exitCode=6
Jul 17 06:27:46.259951 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:46.259516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" event={"ID":"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4","Type":"ContainerDied","Data":"3f954c446a86d706757d46e23c6c72f2db6322928513caea7f9b5940d80ead45"}
Jul 17 06:27:46.259951 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:46.259710    2559 scope.go:117] "RemoveContainer" containerID="3f954c446a86d706757d46e23c6c72f2db6322928513caea7f9b5940d80ead45"
Jul 17 06:27:47.266419 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:47.266327    2559 generic.go:356] "Generic (PLEG): container finished" podID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerID="c20a21cc6e59c36c1dade60aa6e3cc49c595da70fb72680ac9ab496e71e2b0ce" exitCode=6
Jul 17 06:27:47.266890 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:47.266415    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" event={"ID":"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4","Type":"ContainerDied","Data":"c20a21cc6e59c36c1dade60aa6e3cc49c595da70fb72680ac9ab496e71e2b0ce"}
Jul 17 06:27:47.266890 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:47.266475    2559 scope.go:117] "RemoveContainer" containerID="3f954c446a86d706757d46e23c6c72f2db6322928513caea7f9b5940d80ead45"
Jul 17 06:27:47.266890 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:47.266728    2559 scope.go:117] "RemoveContainer" containerID="c20a21cc6e59c36c1dade60aa6e3cc49c595da70fb72680ac9ab496e71e2b0ce"
Jul 17 06:27:47.267091 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:27:47.267010    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-stwks_odh-ai-gateway-infra(d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4"
Jul 17 06:27:48.276434 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:27:48.276402    2559 scope.go:117] "RemoveContainer" containerID="c20a21cc6e59c36c1dade60aa6e3cc49c595da70fb72680ac9ab496e71e2b0ce"
Jul 17 06:27:48.276923 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:27:48.276635    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-stwks_odh-ai-gateway-infra(d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4"
Jul 17 06:28:01.746404 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:01.746371    2559 scope.go:117] "RemoveContainer" containerID="c20a21cc6e59c36c1dade60aa6e3cc49c595da70fb72680ac9ab496e71e2b0ce"
Jul 17 06:28:02.346101 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:02.346008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" event={"ID":"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4","Type":"ContainerStarted","Data":"1180d7505cabbf45f0c4560400c2a2a8482df80ea65f90df51d3e717b5769319"}
Jul 17 06:28:02.362876 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:02.362797    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" podStartSLOduration=18.362773298 podStartE2EDuration="18.362773298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:28:02.360614045 +0000 UTC m=+3307.139096999" watchObservedRunningTime="2026-07-17 06:28:02.362773298 +0000 UTC m=+3307.141256260"
Jul 17 06:28:03.360145 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:03.360107    2559 generic.go:356] "Generic (PLEG): container finished" podID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerID="1180d7505cabbf45f0c4560400c2a2a8482df80ea65f90df51d3e717b5769319" exitCode=6
Jul 17 06:28:03.360666 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:03.360159    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" event={"ID":"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4","Type":"ContainerDied","Data":"1180d7505cabbf45f0c4560400c2a2a8482df80ea65f90df51d3e717b5769319"}
Jul 17 06:28:03.360666 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:03.360232    2559 scope.go:117] "RemoveContainer" containerID="c20a21cc6e59c36c1dade60aa6e3cc49c595da70fb72680ac9ab496e71e2b0ce"
Jul 17 06:28:03.360666 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:03.360543    2559 scope.go:117] "RemoveContainer" containerID="1180d7505cabbf45f0c4560400c2a2a8482df80ea65f90df51d3e717b5769319"
Jul 17 06:28:03.360819 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:03.360774    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-stwks_odh-ai-gateway-infra(d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4"
Jul 17 06:28:03.370550 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:03.370515    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"]
Jul 17 06:28:04.508702 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:04.508674    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:28:04.658616 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:04.658496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4-maas-api-service-ca\") pod \"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4\" (UID: \"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4\") "
Jul 17 06:28:04.658920 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:04.658894    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" (UID: "d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:28:04.759473 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:04.759418    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:05.373445 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:05.373408    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks" event={"ID":"d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4","Type":"ContainerDied","Data":"26f2b87c0d84cd7f8bfbec05818495818a722a27ea59c03bdd91edb040aafde9"}
Jul 17 06:28:05.373645 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:05.373473    2559 scope.go:117] "RemoveContainer" containerID="1180d7505cabbf45f0c4560400c2a2a8482df80ea65f90df51d3e717b5769319"
Jul 17 06:28:05.373645 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:05.373419    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"
Jul 17 06:28:05.396615 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:05.396578    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"]
Jul 17 06:28:05.402335 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:05.402305    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-stwks"]
Jul 17 06:28:05.753053 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:05.753020    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" path="/var/lib/kubelet/pods/d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4/volumes"
Jul 17 06:28:08.744598 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744562    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"]
Jul 17 06:28:08.744998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744942    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.744998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744954    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.744998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744972    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.744998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744978    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.744998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744991    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:28:08.744998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.744997    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0042bba9-a026-4116-9bc4-246f80c7f4d2" containerName="revoke-keys"
Jul 17 06:28:08.745225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.745005    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.745225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.745011    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.745225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.745066    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.745225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.745073    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.745225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.745080    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d0c8d7-6ae0-48ee-8f12-ed8789ccffc4" containerName="revoke-keys"
Jul 17 06:28:08.748125 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.748096    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:08.750708 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.750678    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:28:08.750708 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.750689    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:28:08.755140 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.755110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"]
Jul 17 06:28:08.793282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.793244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d98a31b-82a3-4096-8869-84f7bba25b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84\" (UID: \"8d98a31b-82a3-4096-8869-84f7bba25b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:08.893746 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.893702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d98a31b-82a3-4096-8869-84f7bba25b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84\" (UID: \"8d98a31b-82a3-4096-8869-84f7bba25b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:08.894405 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:08.894385    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d98a31b-82a3-4096-8869-84f7bba25b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84\" (UID: \"8d98a31b-82a3-4096-8869-84f7bba25b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:09.060816 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:09.060716    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:09.201246 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:09.201173    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"]
Jul 17 06:28:09.203912 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:09.203879    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d98a31b_82a3_4096_8869_84f7bba25b66.slice/crio-6a9b207df3f9e2ace8fd40747ad4a6134809bb4c6ca878bd702677321860f3bd WatchSource:0}: Error finding container 6a9b207df3f9e2ace8fd40747ad4a6134809bb4c6ca878bd702677321860f3bd: Status 404 returned error can't find the container with id 6a9b207df3f9e2ace8fd40747ad4a6134809bb4c6ca878bd702677321860f3bd
Jul 17 06:28:09.393040 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:09.393002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" event={"ID":"8d98a31b-82a3-4096-8869-84f7bba25b66","Type":"ContainerStarted","Data":"6a9b207df3f9e2ace8fd40747ad4a6134809bb4c6ca878bd702677321860f3bd"}
Jul 17 06:28:10.399423 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:10.399385    2559 generic.go:356] "Generic (PLEG): container finished" podID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerID="50221ec5540551c651be3a2f4bc415b5d3bb0d22adf71aa8df17157fe50aeab9" exitCode=6
Jul 17 06:28:10.399950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:10.399471    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" event={"ID":"8d98a31b-82a3-4096-8869-84f7bba25b66","Type":"ContainerDied","Data":"50221ec5540551c651be3a2f4bc415b5d3bb0d22adf71aa8df17157fe50aeab9"}
Jul 17 06:28:10.399950 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:10.399685    2559 scope.go:117] "RemoveContainer" containerID="50221ec5540551c651be3a2f4bc415b5d3bb0d22adf71aa8df17157fe50aeab9"
Jul 17 06:28:11.406148 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:11.406059    2559 generic.go:356] "Generic (PLEG): container finished" podID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerID="afa8113ff6338eac647f36c6613f515cccb364f29ee27f5a444cc4597b112209" exitCode=6
Jul 17 06:28:11.406565 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:11.406141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" event={"ID":"8d98a31b-82a3-4096-8869-84f7bba25b66","Type":"ContainerDied","Data":"afa8113ff6338eac647f36c6613f515cccb364f29ee27f5a444cc4597b112209"}
Jul 17 06:28:11.406565 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:11.406194    2559 scope.go:117] "RemoveContainer" containerID="50221ec5540551c651be3a2f4bc415b5d3bb0d22adf71aa8df17157fe50aeab9"
Jul 17 06:28:11.406565 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:11.406436    2559 scope.go:117] "RemoveContainer" containerID="afa8113ff6338eac647f36c6613f515cccb364f29ee27f5a444cc4597b112209"
Jul 17 06:28:11.406681 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:11.406638    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84_odh-ai-gateway-infra(8d98a31b-82a3-4096-8869-84f7bba25b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66"
Jul 17 06:28:12.412638 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:12.412606    2559 scope.go:117] "RemoveContainer" containerID="afa8113ff6338eac647f36c6613f515cccb364f29ee27f5a444cc4597b112209"
Jul 17 06:28:12.413064 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:12.412797    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84_odh-ai-gateway-infra(8d98a31b-82a3-4096-8869-84f7bba25b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66"
Jul 17 06:28:25.089886 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:25.089840    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"]
Jul 17 06:28:25.748644 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:25.748612    2559 scope.go:117] "RemoveContainer" containerID="afa8113ff6338eac647f36c6613f515cccb364f29ee27f5a444cc4597b112209"
Jul 17 06:28:25.925660 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:25.925628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:28:26.483559 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:26.483525    2559 generic.go:356] "Generic (PLEG): container finished" podID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerID="daf8b0734dfa407dddc81a160afa7293115a0246d04a27bfd3800fd67ec84120" exitCode=6
Jul 17 06:28:26.484139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:26.483600    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" event={"ID":"8d98a31b-82a3-4096-8869-84f7bba25b66","Type":"ContainerDied","Data":"daf8b0734dfa407dddc81a160afa7293115a0246d04a27bfd3800fd67ec84120"}
Jul 17 06:28:26.484139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:26.483668    2559 scope.go:117] "RemoveContainer" containerID="afa8113ff6338eac647f36c6613f515cccb364f29ee27f5a444cc4597b112209"
Jul 17 06:28:26.484139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:26.484122    2559 scope.go:117] "RemoveContainer" containerID="daf8b0734dfa407dddc81a160afa7293115a0246d04a27bfd3800fd67ec84120"
Jul 17 06:28:26.484354 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:26.484331    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84_odh-ai-gateway-infra(8d98a31b-82a3-4096-8869-84f7bba25b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66"
Jul 17 06:28:27.038458 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:27.038427    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:28:27.509996 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:27.509961    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"]
Jul 17 06:28:27.652095 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:27.652057    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:27.765311 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:27.765212    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d98a31b-82a3-4096-8869-84f7bba25b66-maas-api-service-ca\") pod \"8d98a31b-82a3-4096-8869-84f7bba25b66\" (UID: \"8d98a31b-82a3-4096-8869-84f7bba25b66\") "
Jul 17 06:28:27.765686 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:27.765657    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d98a31b-82a3-4096-8869-84f7bba25b66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d98a31b-82a3-4096-8869-84f7bba25b66" (UID: "8d98a31b-82a3-4096-8869-84f7bba25b66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:28:27.866089 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:27.866049    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d98a31b-82a3-4096-8869-84f7bba25b66-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:28.497466 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:28.497437    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"
Jul 17 06:28:28.497774 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:28.497435    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84" event={"ID":"8d98a31b-82a3-4096-8869-84f7bba25b66","Type":"ContainerDied","Data":"6a9b207df3f9e2ace8fd40747ad4a6134809bb4c6ca878bd702677321860f3bd"}
Jul 17 06:28:28.497774 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:28.497572    2559 scope.go:117] "RemoveContainer" containerID="daf8b0734dfa407dddc81a160afa7293115a0246d04a27bfd3800fd67ec84120"
Jul 17 06:28:28.522537 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:28.522505    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"]
Jul 17 06:28:28.529013 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:28.528978    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-vsf84"]
Jul 17 06:28:29.575681 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.575637    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7409-140212734153904"]
Jul 17 06:28:29.576221 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576172    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576221 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576192    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576221 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576212    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576221 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576220    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576440 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576237    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576440 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576245    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576440 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576370    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576440 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576383    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.576440 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.576391    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" containerName="revoke-keys"
Jul 17 06:28:29.580712 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.580687    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:29.583800 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.583771    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-n2k4r\""
Jul 17 06:28:29.591796 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.591773    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7409-140212734153904"]
Jul 17 06:28:29.684974 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.684933    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptp9p\" (UniqueName: \"kubernetes.io/projected/580264ed-9c3b-42df-8d0a-a70b56776aaf-kube-api-access-ptp9p\") pod \"test-curl-7409-140212734153904\" (UID: \"580264ed-9c3b-42df-8d0a-a70b56776aaf\") " pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:29.751929 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.751896    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d98a31b-82a3-4096-8869-84f7bba25b66" path="/var/lib/kubelet/pods/8d98a31b-82a3-4096-8869-84f7bba25b66/volumes"
Jul 17 06:28:29.786126 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.786084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptp9p\" (UniqueName: \"kubernetes.io/projected/580264ed-9c3b-42df-8d0a-a70b56776aaf-kube-api-access-ptp9p\") pod \"test-curl-7409-140212734153904\" (UID: \"580264ed-9c3b-42df-8d0a-a70b56776aaf\") " pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:29.794176 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.794141    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptp9p\" (UniqueName: \"kubernetes.io/projected/580264ed-9c3b-42df-8d0a-a70b56776aaf-kube-api-access-ptp9p\") pod \"test-curl-7409-140212734153904\" (UID: \"580264ed-9c3b-42df-8d0a-a70b56776aaf\") " pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:29.890234 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:29.890141    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:30.025926 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:30.025893    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7409-140212734153904"]
Jul 17 06:28:30.028606 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:30.028569    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod580264ed_9c3b_42df_8d0a_a70b56776aaf.slice/crio-3ba33ce2ac6ed42eb003212e658baa3510625f7b20e289261353a4d548b39154 WatchSource:0}: Error finding container 3ba33ce2ac6ed42eb003212e658baa3510625f7b20e289261353a4d548b39154: Status 404 returned error can't find the container with id 3ba33ce2ac6ed42eb003212e658baa3510625f7b20e289261353a4d548b39154
Jul 17 06:28:30.030023 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:30.030001    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 06:28:30.509477 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:30.509446    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7409-140212734153904" event={"ID":"580264ed-9c3b-42df-8d0a-a70b56776aaf","Type":"ContainerStarted","Data":"3ba33ce2ac6ed42eb003212e658baa3510625f7b20e289261353a4d548b39154"}
Jul 17 06:28:31.515452 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:31.515416    2559 generic.go:356] "Generic (PLEG): container finished" podID="580264ed-9c3b-42df-8d0a-a70b56776aaf" containerID="e0d46f1ecf7841442c4029fb8abd56f51a0a6836d92291a74f712d841d20b6a0" exitCode=0
Jul 17 06:28:31.515926 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:31.515513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7409-140212734153904" event={"ID":"580264ed-9c3b-42df-8d0a-a70b56776aaf","Type":"ContainerDied","Data":"e0d46f1ecf7841442c4029fb8abd56f51a0a6836d92291a74f712d841d20b6a0"}
Jul 17 06:28:32.662553 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.662524    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:32.682614 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.682586    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7409-140212734153904_580264ed-9c3b-42df-8d0a-a70b56776aaf/test-curl-7409-140212734153904/0.log"
Jul 17 06:28:32.707784 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.707742    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7409-140212734153904"]
Jul 17 06:28:32.712035 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.711997    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptp9p\" (UniqueName: \"kubernetes.io/projected/580264ed-9c3b-42df-8d0a-a70b56776aaf-kube-api-access-ptp9p\") pod \"580264ed-9c3b-42df-8d0a-a70b56776aaf\" (UID: \"580264ed-9c3b-42df-8d0a-a70b56776aaf\") "
Jul 17 06:28:32.715588 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.715525    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/580264ed-9c3b-42df-8d0a-a70b56776aaf-kube-api-access-ptp9p" (OuterVolumeSpecName: "kube-api-access-ptp9p") pod "580264ed-9c3b-42df-8d0a-a70b56776aaf" (UID: "580264ed-9c3b-42df-8d0a-a70b56776aaf"). InnerVolumeSpecName "kube-api-access-ptp9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:28:32.716906 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.716391    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7409-140212734153904"]
Jul 17 06:28:32.813768 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.813660    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptp9p\" (UniqueName: \"kubernetes.io/projected/580264ed-9c3b-42df-8d0a-a70b56776aaf-kube-api-access-ptp9p\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:32.879751 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.879711    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7409-140212735273840"]
Jul 17 06:28:32.880108 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.880092    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="580264ed-9c3b-42df-8d0a-a70b56776aaf" containerName="test-curl-7409-140212734153904"
Jul 17 06:28:32.880163 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.880109    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="580264ed-9c3b-42df-8d0a-a70b56776aaf" containerName="test-curl-7409-140212734153904"
Jul 17 06:28:32.880196 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.880184    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="580264ed-9c3b-42df-8d0a-a70b56776aaf" containerName="test-curl-7409-140212734153904"
Jul 17 06:28:32.883160 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.883140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:32.887330 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:32.887299    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7409-140212735273840"]
Jul 17 06:28:33.015435 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.015385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dr4z\" (UniqueName: \"kubernetes.io/projected/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0-kube-api-access-2dr4z\") pod \"test-curl-7409-140212735273840\" (UID: \"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0\") " pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:33.116327 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.116222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dr4z\" (UniqueName: \"kubernetes.io/projected/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0-kube-api-access-2dr4z\") pod \"test-curl-7409-140212735273840\" (UID: \"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0\") " pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:33.125408 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.125366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dr4z\" (UniqueName: \"kubernetes.io/projected/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0-kube-api-access-2dr4z\") pod \"test-curl-7409-140212735273840\" (UID: \"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0\") " pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:33.193620 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.193587    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:33.331494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.331466    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7409-140212735273840"]
Jul 17 06:28:33.333961 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:33.333931    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85f7b5c2_ad22_4c93_99d1_9a55b4cf96c0.slice/crio-769ac83a9a5d70e4bc36abdbc3b50289ea570bbfffeea2345a92bbc873c212f9 WatchSource:0}: Error finding container 769ac83a9a5d70e4bc36abdbc3b50289ea570bbfffeea2345a92bbc873c212f9: Status 404 returned error can't find the container with id 769ac83a9a5d70e4bc36abdbc3b50289ea570bbfffeea2345a92bbc873c212f9
Jul 17 06:28:33.525875 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.525837    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:33.526049 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.525889    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ba33ce2ac6ed42eb003212e658baa3510625f7b20e289261353a4d548b39154"
Jul 17 06:28:33.527168 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.527076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7409-140212735273840" event={"ID":"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0","Type":"ContainerStarted","Data":"769ac83a9a5d70e4bc36abdbc3b50289ea570bbfffeea2345a92bbc873c212f9"}
Jul 17 06:28:33.537302 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:33.537270    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7409-140212734153904\" is forbidden: User \"system:node:ip-10-0-128-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-189.ec2.internal' and this object" podUID="580264ed-9c3b-42df-8d0a-a70b56776aaf" pod="opendatahub/test-curl-7409-140212734153904"
Jul 17 06:28:33.752389 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:33.752351    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="580264ed-9c3b-42df-8d0a-a70b56776aaf" path="/var/lib/kubelet/pods/580264ed-9c3b-42df-8d0a-a70b56776aaf/volumes"
Jul 17 06:28:34.533727 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:34.533686    2559 generic.go:356] "Generic (PLEG): container finished" podID="85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0" containerID="d1f5942c940d31d9afe9ac91d7bcb9c85ba461014345ebfd2737fa56fa573738" exitCode=0
Jul 17 06:28:34.533931 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:34.533755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7409-140212735273840" event={"ID":"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0","Type":"ContainerDied","Data":"d1f5942c940d31d9afe9ac91d7bcb9c85ba461014345ebfd2737fa56fa573738"}
Jul 17 06:28:35.682380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.682343    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:35.704797 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.704761    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7409-140212735273840_85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0/test-curl-7409-140212735273840/0.log"
Jul 17 06:28:35.728826 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.728789    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7409-140212735273840"]
Jul 17 06:28:35.732091 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.732056    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7409-140212735273840"]
Jul 17 06:28:35.740419 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.740397    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dr4z\" (UniqueName: \"kubernetes.io/projected/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0-kube-api-access-2dr4z\") pod \"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0\" (UID: \"85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0\") "
Jul 17 06:28:35.742488 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.742463    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0-kube-api-access-2dr4z" (OuterVolumeSpecName: "kube-api-access-2dr4z") pod "85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0" (UID: "85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0"). InnerVolumeSpecName "kube-api-access-2dr4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:28:35.751479 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.751450    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0" path="/var/lib/kubelet/pods/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0/volumes"
Jul 17 06:28:35.841189 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:35.841099    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dr4z\" (UniqueName: \"kubernetes.io/projected/85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0-kube-api-access-2dr4z\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:36.544113 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:36.544067    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735273840"
Jul 17 06:28:36.544113 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:36.544067    2559 scope.go:117] "RemoveContainer" containerID="d1f5942c940d31d9afe9ac91d7bcb9c85ba461014345ebfd2737fa56fa573738"
Jul 17 06:28:40.111971 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.111922    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz" podUID="cc74b36f-4364-49e2-9489-f74de93d7d20" containerName="main" containerID="cri-o://ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312" gracePeriod=45
Jul 17 06:28:40.383743 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.383709    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:28:40.481569 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481519    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-model-cache\") pod \"cc74b36f-4364-49e2-9489-f74de93d7d20\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") "
Jul 17 06:28:40.481569 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-tmp-dir\") pod \"cc74b36f-4364-49e2-9489-f74de93d7d20\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") "
Jul 17 06:28:40.481870 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481615    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-dshm\") pod \"cc74b36f-4364-49e2-9489-f74de93d7d20\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") "
Jul 17 06:28:40.481870 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481680    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-home\") pod \"cc74b36f-4364-49e2-9489-f74de93d7d20\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") "
Jul 17 06:28:40.481870 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481717    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pw7j\" (UniqueName: \"kubernetes.io/projected/cc74b36f-4364-49e2-9489-f74de93d7d20-kube-api-access-2pw7j\") pod \"cc74b36f-4364-49e2-9489-f74de93d7d20\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") "
Jul 17 06:28:40.481870 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481752    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc74b36f-4364-49e2-9489-f74de93d7d20-tls-certs\") pod \"cc74b36f-4364-49e2-9489-f74de93d7d20\" (UID: \"cc74b36f-4364-49e2-9489-f74de93d7d20\") "
Jul 17 06:28:40.482085 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481948    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc74b36f-4364-49e2-9489-f74de93d7d20" (UID: "cc74b36f-4364-49e2-9489-f74de93d7d20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:40.482085 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.481955    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc74b36f-4364-49e2-9489-f74de93d7d20" (UID: "cc74b36f-4364-49e2-9489-f74de93d7d20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:40.482192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.482097    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-model-cache\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:40.482192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.482153    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-tmp-dir\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:40.482192 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.482161    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-home" (OuterVolumeSpecName: "home") pod "cc74b36f-4364-49e2-9489-f74de93d7d20" (UID: "cc74b36f-4364-49e2-9489-f74de93d7d20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:40.483884 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.483844    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-dshm" (OuterVolumeSpecName: "dshm") pod "cc74b36f-4364-49e2-9489-f74de93d7d20" (UID: "cc74b36f-4364-49e2-9489-f74de93d7d20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:40.484190 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.484169    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc74b36f-4364-49e2-9489-f74de93d7d20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc74b36f-4364-49e2-9489-f74de93d7d20" (UID: "cc74b36f-4364-49e2-9489-f74de93d7d20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:28:40.484266 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.484172    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc74b36f-4364-49e2-9489-f74de93d7d20-kube-api-access-2pw7j" (OuterVolumeSpecName: "kube-api-access-2pw7j") pod "cc74b36f-4364-49e2-9489-f74de93d7d20" (UID: "cc74b36f-4364-49e2-9489-f74de93d7d20"). InnerVolumeSpecName "kube-api-access-2pw7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:28:40.566165 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.566131    2559 generic.go:356] "Generic (PLEG): container finished" podID="cc74b36f-4364-49e2-9489-f74de93d7d20" containerID="ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312" exitCode=0
Jul 17 06:28:40.566356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.566193    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz" event={"ID":"cc74b36f-4364-49e2-9489-f74de93d7d20","Type":"ContainerDied","Data":"ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312"}
Jul 17 06:28:40.566356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.566194    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"
Jul 17 06:28:40.566356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.566226    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz" event={"ID":"cc74b36f-4364-49e2-9489-f74de93d7d20","Type":"ContainerDied","Data":"35d279fd0a550f5c30018791e3303d8ec08994ef0ceb263876d3d22cb8fd2afd"}
Jul 17 06:28:40.566356 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.566250    2559 scope.go:117] "RemoveContainer" containerID="ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312"
Jul 17 06:28:40.575876 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.575833    2559 scope.go:117] "RemoveContainer" containerID="ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312"
Jul 17 06:28:40.576222 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:40.576189    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312\": container with ID starting with ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312 not found: ID does not exist" containerID="ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312"
Jul 17 06:28:40.576346 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.576234    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312"} err="failed to get container status \"ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312\": rpc error: code = NotFound desc = could not find container \"ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312\": container with ID starting with ce400a92a72a79db028dbdfd179f102083123cc421290f07e95b2a6017b13312 not found: ID does not exist"
Jul 17 06:28:40.583289 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.583258    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-dshm\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:40.583289 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.583288    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc74b36f-4364-49e2-9489-f74de93d7d20-home\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:40.583457 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.583302    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pw7j\" (UniqueName: \"kubernetes.io/projected/cc74b36f-4364-49e2-9489-f74de93d7d20-kube-api-access-2pw7j\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:40.583457 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.583317    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc74b36f-4364-49e2-9489-f74de93d7d20-tls-certs\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:40.590703 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.590668    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"]
Jul 17 06:28:40.596004 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:40.595977    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e9bcaa/rate-test-model-e9bcaa-kserve-6694cf56b7-lpgmz"]
Jul 17 06:28:41.478585 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.478547    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"]
Jul 17 06:28:41.479032 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.478941    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0" containerName="test-curl-7409-140212735273840"
Jul 17 06:28:41.479032 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.478954    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0" containerName="test-curl-7409-140212735273840"
Jul 17 06:28:41.479032 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.478988    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc74b36f-4364-49e2-9489-f74de93d7d20" containerName="main"
Jul 17 06:28:41.479032 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.478994    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc74b36f-4364-49e2-9489-f74de93d7d20" containerName="main"
Jul 17 06:28:41.479190 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.479057    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f7b5c2-ad22-4c93-99d1-9a55b4cf96c0" containerName="test-curl-7409-140212735273840"
Jul 17 06:28:41.479190 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.479064    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc74b36f-4364-49e2-9489-f74de93d7d20" containerName="main"
Jul 17 06:28:41.483283 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.483262    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:28:41.486073 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.486037    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:28:41.486073 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.486057    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:28:41.491503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.491278    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"]
Jul 17 06:28:41.592761 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.592724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f21485ca-3ef7-467c-b2f7-75c846fbc7ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-qvvd2\" (UID: \"f21485ca-3ef7-467c-b2f7-75c846fbc7ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:28:41.611729 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.611693    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7409-140212735274848"]
Jul 17 06:28:41.614868 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.614841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:41.621596 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.621514    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-n2k4r\""
Jul 17 06:28:41.632703 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.632667    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7409-140212735274848"]
Jul 17 06:28:41.693909 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.693842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f21485ca-3ef7-467c-b2f7-75c846fbc7ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-qvvd2\" (UID: \"f21485ca-3ef7-467c-b2f7-75c846fbc7ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:28:41.694104 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.693963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkr29\" (UniqueName: \"kubernetes.io/projected/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5-kube-api-access-tkr29\") pod \"test-curl-7409-140212735274848\" (UID: \"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5\") " pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:41.694537 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.694516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f21485ca-3ef7-467c-b2f7-75c846fbc7ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3232801c-qvvd2\" (UID: \"f21485ca-3ef7-467c-b2f7-75c846fbc7ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:28:41.756382 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.756284    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc74b36f-4364-49e2-9489-f74de93d7d20" path="/var/lib/kubelet/pods/cc74b36f-4364-49e2-9489-f74de93d7d20/volumes"
Jul 17 06:28:41.794810 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.794771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkr29\" (UniqueName: \"kubernetes.io/projected/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5-kube-api-access-tkr29\") pod \"test-curl-7409-140212735274848\" (UID: \"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5\") " pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:41.795607 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.795583    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:28:41.805343 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.805318    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkr29\" (UniqueName: \"kubernetes.io/projected/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5-kube-api-access-tkr29\") pod \"test-curl-7409-140212735274848\" (UID: \"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5\") " pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:41.924777 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.924738    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:41.937055 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:41.937025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"]
Jul 17 06:28:41.940820 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:41.940786    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21485ca_3ef7_467c_b2f7_75c846fbc7ab.slice/crio-f6dc631ae8de2774976bf525bf63d9a9c96e93a8d7cc9b0c98176132e514c265 WatchSource:0}: Error finding container f6dc631ae8de2774976bf525bf63d9a9c96e93a8d7cc9b0c98176132e514c265: Status 404 returned error can't find the container with id f6dc631ae8de2774976bf525bf63d9a9c96e93a8d7cc9b0c98176132e514c265
Jul 17 06:28:42.067473 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.067401    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7409-140212735274848"]
Jul 17 06:28:42.069869 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:42.069821    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod116b2e23_2ac5_4eca_abd2_eb73ed69a8c5.slice/crio-8414a54080b241f48253d647eed56112dacb37d645285656ac694d750fc5b6d6 WatchSource:0}: Error finding container 8414a54080b241f48253d647eed56112dacb37d645285656ac694d750fc5b6d6: Status 404 returned error can't find the container with id 8414a54080b241f48253d647eed56112dacb37d645285656ac694d750fc5b6d6
Jul 17 06:28:42.578934 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.578799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" event={"ID":"f21485ca-3ef7-467c-b2f7-75c846fbc7ab","Type":"ContainerStarted","Data":"1dd7cf0c19a6719f5c3304a692b1a4dd11a655e49c614ce597d0f3571e865444"}
Jul 17 06:28:42.578934 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.578877    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" event={"ID":"f21485ca-3ef7-467c-b2f7-75c846fbc7ab","Type":"ContainerStarted","Data":"f6dc631ae8de2774976bf525bf63d9a9c96e93a8d7cc9b0c98176132e514c265"}
Jul 17 06:28:42.581132 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.581096    2559 generic.go:356] "Generic (PLEG): container finished" podID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" containerID="ddd7acfacf6e2b20efc2f80f5b88f3f561cd41f6a5bce64b1e622cfce895f635" exitCode=0
Jul 17 06:28:42.581277 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.581185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7409-140212735274848" event={"ID":"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5","Type":"ContainerDied","Data":"ddd7acfacf6e2b20efc2f80f5b88f3f561cd41f6a5bce64b1e622cfce895f635"}
Jul 17 06:28:42.581277 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.581222    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7409-140212735274848" event={"ID":"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5","Type":"ContainerStarted","Data":"8414a54080b241f48253d647eed56112dacb37d645285656ac694d750fc5b6d6"}
Jul 17 06:28:42.595199 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:42.595145    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" podStartSLOduration=1.595125866 podStartE2EDuration="1.595125866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:28:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:28:42.593287597 +0000 UTC m=+3347.371770557" watchObservedRunningTime="2026-07-17 06:28:42.595125866 +0000 UTC m=+3347.373608828"
Jul 17 06:28:43.587601 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.587569    2559 generic.go:356] "Generic (PLEG): container finished" podID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerID="1dd7cf0c19a6719f5c3304a692b1a4dd11a655e49c614ce597d0f3571e865444" exitCode=6
Jul 17 06:28:43.588109 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.587653    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" event={"ID":"f21485ca-3ef7-467c-b2f7-75c846fbc7ab","Type":"ContainerDied","Data":"1dd7cf0c19a6719f5c3304a692b1a4dd11a655e49c614ce597d0f3571e865444"}
Jul 17 06:28:43.588109 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.587874    2559 scope.go:117] "RemoveContainer" containerID="1dd7cf0c19a6719f5c3304a692b1a4dd11a655e49c614ce597d0f3571e865444"
Jul 17 06:28:43.727739 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.727714    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:43.748171 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.748142    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7409-140212735274848_116b2e23-2ac5-4eca-abd2-eb73ed69a8c5/test-curl-7409-140212735274848/0.log"
Jul 17 06:28:43.774658 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.774627    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7409-140212735274848"]
Jul 17 06:28:43.778402 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.778378    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7409-140212735274848"]
Jul 17 06:28:43.811269 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.811227    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkr29\" (UniqueName: \"kubernetes.io/projected/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5-kube-api-access-tkr29\") pod \"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5\" (UID: \"116b2e23-2ac5-4eca-abd2-eb73ed69a8c5\") "
Jul 17 06:28:43.862570 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.862475    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5-kube-api-access-tkr29" (OuterVolumeSpecName: "kube-api-access-tkr29") pod "116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" (UID: "116b2e23-2ac5-4eca-abd2-eb73ed69a8c5"). InnerVolumeSpecName "kube-api-access-tkr29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:28:43.912555 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.912520    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkr29\" (UniqueName: \"kubernetes.io/projected/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5-kube-api-access-tkr29\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:43.955614 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.955570    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"]
Jul 17 06:28:43.956196 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.956172    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" containerName="test-curl-7409-140212735274848"
Jul 17 06:28:43.956249 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.956200    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" containerName="test-curl-7409-140212735274848"
Jul 17 06:28:43.956288 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.956278    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" containerName="test-curl-7409-140212735274848"
Jul 17 06:28:43.959785 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.959764    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:43.965309 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:43.965237    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"]
Jul 17 06:28:44.114386 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.114268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c9ca9b-504b-48d8-849f-61fc762b2146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz\" (UID: \"c8c9ca9b-504b-48d8-849f-61fc762b2146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:44.215264 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.215225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c9ca9b-504b-48d8-849f-61fc762b2146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz\" (UID: \"c8c9ca9b-504b-48d8-849f-61fc762b2146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:44.215964 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.215940    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c9ca9b-504b-48d8-849f-61fc762b2146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz\" (UID: \"c8c9ca9b-504b-48d8-849f-61fc762b2146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:44.276320 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.276268    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:44.424595 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.424567    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"]
Jul 17 06:28:44.426718 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:44.426679    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c9ca9b_504b_48d8_849f_61fc762b2146.slice/crio-3b38fd6999b9451958f3ed73c637712ee68ee9ce96ef7bef3583457e80cba58d WatchSource:0}: Error finding container 3b38fd6999b9451958f3ed73c637712ee68ee9ce96ef7bef3583457e80cba58d: Status 404 returned error can't find the container with id 3b38fd6999b9451958f3ed73c637712ee68ee9ce96ef7bef3583457e80cba58d
Jul 17 06:28:44.476847 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.476815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:28:44.578242 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.578195    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:28:44.594816 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.594776    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz" event={"ID":"c8c9ca9b-504b-48d8-849f-61fc762b2146","Type":"ContainerStarted","Data":"3b38fd6999b9451958f3ed73c637712ee68ee9ce96ef7bef3583457e80cba58d"}
Jul 17 06:28:44.596631 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.596597    2559 generic.go:356] "Generic (PLEG): container finished" podID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerID="ec891704db51936fe510d10960e05e585be96f6c1468836b2e30eaf8c47e67b1" exitCode=6
Jul 17 06:28:44.596806 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.596687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" event={"ID":"f21485ca-3ef7-467c-b2f7-75c846fbc7ab","Type":"ContainerDied","Data":"ec891704db51936fe510d10960e05e585be96f6c1468836b2e30eaf8c47e67b1"}
Jul 17 06:28:44.596806 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.596748    2559 scope.go:117] "RemoveContainer" containerID="1dd7cf0c19a6719f5c3304a692b1a4dd11a655e49c614ce597d0f3571e865444"
Jul 17 06:28:44.597116 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.597089    2559 scope.go:117] "RemoveContainer" containerID="ec891704db51936fe510d10960e05e585be96f6c1468836b2e30eaf8c47e67b1"
Jul 17 06:28:44.597401 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:44.597370    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-qvvd2_odh-ai-gateway-infra(f21485ca-3ef7-467c-b2f7-75c846fbc7ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab"
Jul 17 06:28:44.598279 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.598148    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8414a54080b241f48253d647eed56112dacb37d645285656ac694d750fc5b6d6"
Jul 17 06:28:44.598279 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.598178    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:44.617000 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:44.616912    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7409-140212735274848\" is forbidden: User \"system:node:ip-10-0-128-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-189.ec2.internal' and this object" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:44.763240 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.763214    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"]
Jul 17 06:28:44.768270 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.768244    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:28:44.770792 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:44.770744    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7409-140212735274848\" is forbidden: User \"system:node:ip-10-0-128-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-189.ec2.internal' and this object" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:44.772954 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.772932    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"]
Jul 17 06:28:44.922517 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:44.922418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7e3c22a-7d3e-42b8-872e-c9bbae150307-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n\" (UID: \"c7e3c22a-7d3e-42b8-872e-c9bbae150307\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:28:45.023875 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.023824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7e3c22a-7d3e-42b8-872e-c9bbae150307-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n\" (UID: \"c7e3c22a-7d3e-42b8-872e-c9bbae150307\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:28:45.024491 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.024465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7e3c22a-7d3e-42b8-872e-c9bbae150307-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n\" (UID: \"c7e3c22a-7d3e-42b8-872e-c9bbae150307\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:28:45.081907 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.081876    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:28:45.223503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.223475    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"]
Jul 17 06:28:45.226173 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:45.226139    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7e3c22a_7d3e_42b8_872e_c9bbae150307.slice/crio-3482db3684bc885c12017fed68fa74eed19c31b176c996222baba44dbdb61a25 WatchSource:0}: Error finding container 3482db3684bc885c12017fed68fa74eed19c31b176c996222baba44dbdb61a25: Status 404 returned error can't find the container with id 3482db3684bc885c12017fed68fa74eed19c31b176c996222baba44dbdb61a25
Jul 17 06:28:45.606173 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.606138    2559 scope.go:117] "RemoveContainer" containerID="ec891704db51936fe510d10960e05e585be96f6c1468836b2e30eaf8c47e67b1"
Jul 17 06:28:45.606676 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:45.606390    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-qvvd2_odh-ai-gateway-infra(f21485ca-3ef7-467c-b2f7-75c846fbc7ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab"
Jul 17 06:28:45.607507 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.607482    2559 generic.go:356] "Generic (PLEG): container finished" podID="c8c9ca9b-504b-48d8-849f-61fc762b2146" containerID="c60783e90e73d4f33e3f3cbb722adebe1ff3db3792682d6b634f9e784da32b9f" exitCode=0
Jul 17 06:28:45.607622 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.607540    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz" event={"ID":"c8c9ca9b-504b-48d8-849f-61fc762b2146","Type":"ContainerDied","Data":"c60783e90e73d4f33e3f3cbb722adebe1ff3db3792682d6b634f9e784da32b9f"}
Jul 17 06:28:45.608712 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.608691    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" event={"ID":"c7e3c22a-7d3e-42b8-872e-c9bbae150307","Type":"ContainerStarted","Data":"3482db3684bc885c12017fed68fa74eed19c31b176c996222baba44dbdb61a25"}
Jul 17 06:28:45.624244 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:45.624203    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7409-140212735274848\" is forbidden: User \"system:node:ip-10-0-128-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-189.ec2.internal' and this object" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:45.640345 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:45.640307    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7409-140212735274848\" is forbidden: User \"system:node:ip-10-0-128-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-189.ec2.internal' and this object" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:45.751080 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:45.751031    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7409-140212735274848\" is forbidden: User \"system:node:ip-10-0-128-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-189.ec2.internal' and this object" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" pod="opendatahub/test-curl-7409-140212735274848"
Jul 17 06:28:45.754177 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:45.754141    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="116b2e23-2ac5-4eca-abd2-eb73ed69a8c5" path="/var/lib/kubelet/pods/116b2e23-2ac5-4eca-abd2-eb73ed69a8c5/volumes"
Jul 17 06:28:46.615648 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.615612    2559 generic.go:356] "Generic (PLEG): container finished" podID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerID="104887f0c14910b263b4e01400a7034abf5a0d993e3a0440fd6b3cb066253069" exitCode=6
Jul 17 06:28:46.616160 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.615707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" event={"ID":"c7e3c22a-7d3e-42b8-872e-c9bbae150307","Type":"ContainerDied","Data":"104887f0c14910b263b4e01400a7034abf5a0d993e3a0440fd6b3cb066253069"}
Jul 17 06:28:46.616160 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.615975    2559 scope.go:117] "RemoveContainer" containerID="104887f0c14910b263b4e01400a7034abf5a0d993e3a0440fd6b3cb066253069"
Jul 17 06:28:46.768484 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.768452    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:46.841923 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.841876    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c9ca9b-504b-48d8-849f-61fc762b2146-maas-api-service-ca\") pod \"c8c9ca9b-504b-48d8-849f-61fc762b2146\" (UID: \"c8c9ca9b-504b-48d8-849f-61fc762b2146\") "
Jul 17 06:28:46.842253 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.842228    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8c9ca9b-504b-48d8-849f-61fc762b2146-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8c9ca9b-504b-48d8-849f-61fc762b2146" (UID: "c8c9ca9b-504b-48d8-849f-61fc762b2146"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:28:46.943157 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:46.943118    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c9ca9b-504b-48d8-849f-61fc762b2146-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:28:47.622702 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.622604    2559 generic.go:356] "Generic (PLEG): container finished" podID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerID="8b34be64fe04855ec300279d7b2d4ad53d39781c963d2ab966281211ccd5647a" exitCode=6
Jul 17 06:28:47.623220 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.622688    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" event={"ID":"c7e3c22a-7d3e-42b8-872e-c9bbae150307","Type":"ContainerDied","Data":"8b34be64fe04855ec300279d7b2d4ad53d39781c963d2ab966281211ccd5647a"}
Jul 17 06:28:47.623220 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.622762    2559 scope.go:117] "RemoveContainer" containerID="104887f0c14910b263b4e01400a7034abf5a0d993e3a0440fd6b3cb066253069"
Jul 17 06:28:47.623220 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.623018    2559 scope.go:117] "RemoveContainer" containerID="8b34be64fe04855ec300279d7b2d4ad53d39781c963d2ab966281211ccd5647a"
Jul 17 06:28:47.623386 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:47.623287    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n_odh-ai-gateway-infra(c7e3c22a-7d3e-42b8-872e-c9bbae150307)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307"
Jul 17 06:28:47.624400 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.624373    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz" event={"ID":"c8c9ca9b-504b-48d8-849f-61fc762b2146","Type":"ContainerDied","Data":"3b38fd6999b9451958f3ed73c637712ee68ee9ce96ef7bef3583457e80cba58d"}
Jul 17 06:28:47.624520 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.624409    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz"
Jul 17 06:28:47.624520 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:47.624417    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b38fd6999b9451958f3ed73c637712ee68ee9ce96ef7bef3583457e80cba58d"
Jul 17 06:28:48.632375 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:48.632350    2559 scope.go:117] "RemoveContainer" containerID="8b34be64fe04855ec300279d7b2d4ad53d39781c963d2ab966281211ccd5647a"
Jul 17 06:28:48.632887 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:28:48.632616    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n_odh-ai-gateway-infra(c7e3c22a-7d3e-42b8-872e-c9bbae150307)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307"
Jul 17 06:28:54.587700 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:54.587662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:28:58.794410 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.794372    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"]
Jul 17 06:28:58.794963 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.794740    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c9ca9b-504b-48d8-849f-61fc762b2146" containerName="revoke-keys"
Jul 17 06:28:58.794963 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.794751    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c9ca9b-504b-48d8-849f-61fc762b2146" containerName="revoke-keys"
Jul 17 06:28:58.794963 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.794821    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c9ca9b-504b-48d8-849f-61fc762b2146" containerName="revoke-keys"
Jul 17 06:28:58.799587 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.799568    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:28:58.805225 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.805197    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"]
Jul 17 06:28:58.856073 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.856029    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977756a1-e940-40ed-8cf9-c41e8b598867-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87\" (UID: \"977756a1-e940-40ed-8cf9-c41e8b598867\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:28:58.957268 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.957228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977756a1-e940-40ed-8cf9-c41e8b598867-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87\" (UID: \"977756a1-e940-40ed-8cf9-c41e8b598867\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:28:58.957985 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:58.957959    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977756a1-e940-40ed-8cf9-c41e8b598867-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87\" (UID: \"977756a1-e940-40ed-8cf9-c41e8b598867\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:28:59.072515 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:59.072428    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:28:59.110881 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:59.110837    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:28:59.250360 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:59.250332    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"]
Jul 17 06:28:59.252268 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:28:59.252237    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977756a1_e940_40ed_8cf9_c41e8b598867.slice/crio-83b92bedee691ba86c32fa7da9d505991015180405219b4c39a4bef140be49fb WatchSource:0}: Error finding container 83b92bedee691ba86c32fa7da9d505991015180405219b4c39a4bef140be49fb: Status 404 returned error can't find the container with id 83b92bedee691ba86c32fa7da9d505991015180405219b4c39a4bef140be49fb
Jul 17 06:28:59.689735 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:59.689691    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" event={"ID":"977756a1-e940-40ed-8cf9-c41e8b598867","Type":"ContainerStarted","Data":"27f0452cd69c3207d6246e7b4f55a5d402e8ec3e36ee06e151366ab2a249a04a"}
Jul 17 06:28:59.689985 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:59.689796    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" event={"ID":"977756a1-e940-40ed-8cf9-c41e8b598867","Type":"ContainerStarted","Data":"83b92bedee691ba86c32fa7da9d505991015180405219b4c39a4bef140be49fb"}
Jul 17 06:28:59.705202 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:28:59.705136    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" podStartSLOduration=1.705118827 podStartE2EDuration="1.705118827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:28:59.703147132 +0000 UTC m=+3364.481630091" watchObservedRunningTime="2026-07-17 06:28:59.705118827 +0000 UTC m=+3364.483601795"
Jul 17 06:29:00.695770 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:00.695734    2559 generic.go:356] "Generic (PLEG): container finished" podID="977756a1-e940-40ed-8cf9-c41e8b598867" containerID="27f0452cd69c3207d6246e7b4f55a5d402e8ec3e36ee06e151366ab2a249a04a" exitCode=6
Jul 17 06:29:00.696237 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:00.695822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" event={"ID":"977756a1-e940-40ed-8cf9-c41e8b598867","Type":"ContainerDied","Data":"27f0452cd69c3207d6246e7b4f55a5d402e8ec3e36ee06e151366ab2a249a04a"}
Jul 17 06:29:00.696237 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:00.696067    2559 scope.go:117] "RemoveContainer" containerID="27f0452cd69c3207d6246e7b4f55a5d402e8ec3e36ee06e151366ab2a249a04a"
Jul 17 06:29:00.746236 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:00.746203    2559 scope.go:117] "RemoveContainer" containerID="ec891704db51936fe510d10960e05e585be96f6c1468836b2e30eaf8c47e67b1"
Jul 17 06:29:01.705746 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.705655    2559 generic.go:356] "Generic (PLEG): container finished" podID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerID="902df4c7cc0705e79c117b36addded0b987166ddbf1edc36f73690e72b1f3c60" exitCode=6
Jul 17 06:29:01.705746 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.705733    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" event={"ID":"f21485ca-3ef7-467c-b2f7-75c846fbc7ab","Type":"ContainerDied","Data":"902df4c7cc0705e79c117b36addded0b987166ddbf1edc36f73690e72b1f3c60"}
Jul 17 06:29:01.706276 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.705804    2559 scope.go:117] "RemoveContainer" containerID="ec891704db51936fe510d10960e05e585be96f6c1468836b2e30eaf8c47e67b1"
Jul 17 06:29:01.706276 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.706156    2559 scope.go:117] "RemoveContainer" containerID="902df4c7cc0705e79c117b36addded0b987166ddbf1edc36f73690e72b1f3c60"
Jul 17 06:29:01.706429 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:01.706401    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3232801c-qvvd2_odh-ai-gateway-infra(f21485ca-3ef7-467c-b2f7-75c846fbc7ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab"
Jul 17 06:29:01.707719 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.707698    2559 generic.go:356] "Generic (PLEG): container finished" podID="977756a1-e940-40ed-8cf9-c41e8b598867" containerID="8c084c0eca830c9a157778ea5411c1aade1dd94c826bcbdac44a1a26d3d2a4b4" exitCode=6
Jul 17 06:29:01.707805 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.707782    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" event={"ID":"977756a1-e940-40ed-8cf9-c41e8b598867","Type":"ContainerDied","Data":"8c084c0eca830c9a157778ea5411c1aade1dd94c826bcbdac44a1a26d3d2a4b4"}
Jul 17 06:29:01.707976 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.707963    2559 scope.go:117] "RemoveContainer" containerID="8c084c0eca830c9a157778ea5411c1aade1dd94c826bcbdac44a1a26d3d2a4b4"
Jul 17 06:29:01.708174 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:01.708159    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87_odh-ai-gateway-infra(977756a1-e940-40ed-8cf9-c41e8b598867)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" podUID="977756a1-e940-40ed-8cf9-c41e8b598867"
Jul 17 06:29:01.717035 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:01.717013    2559 scope.go:117] "RemoveContainer" containerID="27f0452cd69c3207d6246e7b4f55a5d402e8ec3e36ee06e151366ab2a249a04a"
Jul 17 06:29:02.715547 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:02.715521    2559 scope.go:117] "RemoveContainer" containerID="8c084c0eca830c9a157778ea5411c1aade1dd94c826bcbdac44a1a26d3d2a4b4"
Jul 17 06:29:02.715980 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:02.715757    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87_odh-ai-gateway-infra(977756a1-e940-40ed-8cf9-c41e8b598867)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" podUID="977756a1-e940-40ed-8cf9-c41e8b598867"
Jul 17 06:29:02.742522 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:02.742486    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"]
Jul 17 06:29:02.746667 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:02.746645    2559 scope.go:117] "RemoveContainer" containerID="8b34be64fe04855ec300279d7b2d4ad53d39781c963d2ab966281211ccd5647a"
Jul 17 06:29:02.889556 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:02.889522    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:29:02.992054 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:02.992016    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f21485ca-3ef7-467c-b2f7-75c846fbc7ab-maas-api-service-ca\") pod \"f21485ca-3ef7-467c-b2f7-75c846fbc7ab\" (UID: \"f21485ca-3ef7-467c-b2f7-75c846fbc7ab\") "
Jul 17 06:29:02.992360 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:02.992335    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f21485ca-3ef7-467c-b2f7-75c846fbc7ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f21485ca-3ef7-467c-b2f7-75c846fbc7ab" (UID: "f21485ca-3ef7-467c-b2f7-75c846fbc7ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:29:03.092917 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.092881    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f21485ca-3ef7-467c-b2f7-75c846fbc7ab-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:29:03.129699 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.129657    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"]
Jul 17 06:29:03.130090 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.130075    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:03.130146 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.130093    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:03.130146 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.130110    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:03.130146 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.130116    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:03.130241 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.130185    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:03.130241 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.130198    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:03.133278 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.133260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:03.141417 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.141387    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"]
Jul 17 06:29:03.193431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.193394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da6b6ee9-a785-4971-8c23-90083167d9b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4\" (UID: \"da6b6ee9-a785-4971-8c23-90083167d9b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:03.294646 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.294542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da6b6ee9-a785-4971-8c23-90083167d9b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4\" (UID: \"da6b6ee9-a785-4971-8c23-90083167d9b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:03.295205 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.295185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da6b6ee9-a785-4971-8c23-90083167d9b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4\" (UID: \"da6b6ee9-a785-4971-8c23-90083167d9b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:03.444715 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.444674    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:03.582177 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.582142    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"]
Jul 17 06:29:03.584751 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:29:03.584722    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6b6ee9_a785_4971_8c23_90083167d9b5.slice/crio-04a0a4756dda3b92b22769c70b7de9bf5507a37348ce6c6773e29c122b25c9b4 WatchSource:0}: Error finding container 04a0a4756dda3b92b22769c70b7de9bf5507a37348ce6c6773e29c122b25c9b4: Status 404 returned error can't find the container with id 04a0a4756dda3b92b22769c70b7de9bf5507a37348ce6c6773e29c122b25c9b4
Jul 17 06:29:03.721767 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.721728    2559 generic.go:356] "Generic (PLEG): container finished" podID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerID="411a888e0f34eae8085c48b362b2323fcbc1fb1d0ee966f80d5d5fc2229ed2f5" exitCode=6
Jul 17 06:29:03.722251 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.721799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" event={"ID":"c7e3c22a-7d3e-42b8-872e-c9bbae150307","Type":"ContainerDied","Data":"411a888e0f34eae8085c48b362b2323fcbc1fb1d0ee966f80d5d5fc2229ed2f5"}
Jul 17 06:29:03.722251 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.721911    2559 scope.go:117] "RemoveContainer" containerID="8b34be64fe04855ec300279d7b2d4ad53d39781c963d2ab966281211ccd5647a"
Jul 17 06:29:03.722423 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.722257    2559 scope.go:117] "RemoveContainer" containerID="411a888e0f34eae8085c48b362b2323fcbc1fb1d0ee966f80d5d5fc2229ed2f5"
Jul 17 06:29:03.722540 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:03.722515    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n_odh-ai-gateway-infra(c7e3c22a-7d3e-42b8-872e-c9bbae150307)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307"
Jul 17 06:29:03.723204 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.723184    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"
Jul 17 06:29:03.723328 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.723209    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2" event={"ID":"f21485ca-3ef7-467c-b2f7-75c846fbc7ab","Type":"ContainerDied","Data":"f6dc631ae8de2774976bf525bf63d9a9c96e93a8d7cc9b0c98176132e514c265"}
Jul 17 06:29:03.724494 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.724472    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" event={"ID":"da6b6ee9-a785-4971-8c23-90083167d9b5","Type":"ContainerStarted","Data":"04a0a4756dda3b92b22769c70b7de9bf5507a37348ce6c6773e29c122b25c9b4"}
Jul 17 06:29:03.734751 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.734727    2559 scope.go:117] "RemoveContainer" containerID="902df4c7cc0705e79c117b36addded0b987166ddbf1edc36f73690e72b1f3c60"
Jul 17 06:29:03.751732 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.751700    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"]
Jul 17 06:29:03.755190 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:03.755161    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3232801c-qvvd2"]
Jul 17 06:29:04.733565 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.733534    2559 generic.go:356] "Generic (PLEG): container finished" podID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerID="c745b5513f9235557a4f9e8c33941699a0e04a67c6b58e32253bab11e8d35003" exitCode=6
Jul 17 06:29:04.734007 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.733583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" event={"ID":"da6b6ee9-a785-4971-8c23-90083167d9b5","Type":"ContainerDied","Data":"c745b5513f9235557a4f9e8c33941699a0e04a67c6b58e32253bab11e8d35003"}
Jul 17 06:29:04.734007 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.733791    2559 scope.go:117] "RemoveContainer" containerID="c745b5513f9235557a4f9e8c33941699a0e04a67c6b58e32253bab11e8d35003"
Jul 17 06:29:04.747774 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.747747    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"]
Jul 17 06:29:04.894301 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.894276    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:29:04.910393 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.910048    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7e3c22a-7d3e-42b8-872e-c9bbae150307-maas-api-service-ca\") pod \"c7e3c22a-7d3e-42b8-872e-c9bbae150307\" (UID: \"c7e3c22a-7d3e-42b8-872e-c9bbae150307\") "
Jul 17 06:29:04.910574 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:04.910465    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7e3c22a-7d3e-42b8-872e-c9bbae150307-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7e3c22a-7d3e-42b8-872e-c9bbae150307" (UID: "c7e3c22a-7d3e-42b8-872e-c9bbae150307"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:29:05.011045 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.010961    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7e3c22a-7d3e-42b8-872e-c9bbae150307-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:29:05.101721 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.101686    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:05.169704 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.169670    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:05.741850 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.741814    2559 generic.go:356] "Generic (PLEG): container finished" podID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerID="b3158bb50f70ac97fcbe2a7fa59329887dd836ca626787a1a0f42f31104d7fa1" exitCode=6
Jul 17 06:29:05.742386 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.741895    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" event={"ID":"da6b6ee9-a785-4971-8c23-90083167d9b5","Type":"ContainerDied","Data":"b3158bb50f70ac97fcbe2a7fa59329887dd836ca626787a1a0f42f31104d7fa1"}
Jul 17 06:29:05.742386 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.741973    2559 scope.go:117] "RemoveContainer" containerID="c745b5513f9235557a4f9e8c33941699a0e04a67c6b58e32253bab11e8d35003"
Jul 17 06:29:05.742386 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.742225    2559 scope.go:117] "RemoveContainer" containerID="b3158bb50f70ac97fcbe2a7fa59329887dd836ca626787a1a0f42f31104d7fa1"
Jul 17 06:29:05.742620 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:05.742494    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4_odh-ai-gateway-infra(da6b6ee9-a785-4971-8c23-90083167d9b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5"
Jul 17 06:29:05.743556 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.743529    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"
Jul 17 06:29:05.743673 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.743530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n" event={"ID":"c7e3c22a-7d3e-42b8-872e-c9bbae150307","Type":"ContainerDied","Data":"3482db3684bc885c12017fed68fa74eed19c31b176c996222baba44dbdb61a25"}
Jul 17 06:29:05.756742 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.756691    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" path="/var/lib/kubelet/pods/f21485ca-3ef7-467c-b2f7-75c846fbc7ab/volumes"
Jul 17 06:29:05.760644 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.760620    2559 scope.go:117] "RemoveContainer" containerID="411a888e0f34eae8085c48b362b2323fcbc1fb1d0ee966f80d5d5fc2229ed2f5"
Jul 17 06:29:05.786048 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.786000    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"]
Jul 17 06:29:05.790200 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:05.790136    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81346fa0-dkl5n"]
Jul 17 06:29:06.751042 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:06.751012    2559 scope.go:117] "RemoveContainer" containerID="b3158bb50f70ac97fcbe2a7fa59329887dd836ca626787a1a0f42f31104d7fa1"
Jul 17 06:29:06.751484 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:06.751201    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4_odh-ai-gateway-infra(da6b6ee9-a785-4971-8c23-90083167d9b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5"
Jul 17 06:29:06.820810 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:06.820776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:07.752013 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:07.751980    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" path="/var/lib/kubelet/pods/c7e3c22a-7d3e-42b8-872e-c9bbae150307/volumes"
Jul 17 06:29:09.755205 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:09.755176    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:09.971152 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:09.971116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:10.094780 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.094696    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"]
Jul 17 06:29:10.095080 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095067    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095081    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095089    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095096    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095102    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095107    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095123    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095127 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095128    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095189    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21485ca-3ef7-467c-b2f7-75c846fbc7ab" containerName="revoke-keys"
Jul 17 06:29:10.095347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095196    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095203    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.095347 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.095211    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e3c22a-7d3e-42b8-872e-c9bbae150307" containerName="revoke-keys"
Jul 17 06:29:10.098100 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.098076    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:10.106952 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.106925    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"]
Jul 17 06:29:10.158873 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.158823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac47d58-c5c3-4f2e-8a68-dc3c49b71483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt\" (UID: \"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:10.259984 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.259943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac47d58-c5c3-4f2e-8a68-dc3c49b71483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt\" (UID: \"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:10.260634 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.260612    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac47d58-c5c3-4f2e-8a68-dc3c49b71483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt\" (UID: \"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:10.410629 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.410535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:10.551164 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.551134    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"]
Jul 17 06:29:10.552915 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:29:10.552878    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac47d58_c5c3_4f2e_8a68_dc3c49b71483.slice/crio-72f1354627a6c97ed46b02384e652cf59cf549ca1d81a3554c7b4938dcdaedad WatchSource:0}: Error finding container 72f1354627a6c97ed46b02384e652cf59cf549ca1d81a3554c7b4938dcdaedad: Status 404 returned error can't find the container with id 72f1354627a6c97ed46b02384e652cf59cf549ca1d81a3554c7b4938dcdaedad
Jul 17 06:29:10.657317 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.657287    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:10.757270 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.757235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:10.771458 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:10.771427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt" event={"ID":"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483","Type":"ContainerStarted","Data":"72f1354627a6c97ed46b02384e652cf59cf549ca1d81a3554c7b4938dcdaedad"}
Jul 17 06:29:11.778364 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:11.778321    2559 generic.go:356] "Generic (PLEG): container finished" podID="4ac47d58-c5c3-4f2e-8a68-dc3c49b71483" containerID="3b584e23b701221f04bb4d0c65c4bd709e9b69b110ac8d8523a649e4a46d69e5" exitCode=0
Jul 17 06:29:11.778792 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:11.778361    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt" event={"ID":"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483","Type":"ContainerDied","Data":"3b584e23b701221f04bb4d0c65c4bd709e9b69b110ac8d8523a649e4a46d69e5"}
Jul 17 06:29:12.923050 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:12.923022    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:12.984218 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:12.984184    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac47d58-c5c3-4f2e-8a68-dc3c49b71483-maas-api-service-ca\") pod \"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483\" (UID: \"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483\") "
Jul 17 06:29:12.984568 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:12.984543    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ac47d58-c5c3-4f2e-8a68-dc3c49b71483-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ac47d58-c5c3-4f2e-8a68-dc3c49b71483" (UID: "4ac47d58-c5c3-4f2e-8a68-dc3c49b71483"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:29:13.085241 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:13.085151    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac47d58-c5c3-4f2e-8a68-dc3c49b71483-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:29:13.790181 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:13.790140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt" event={"ID":"4ac47d58-c5c3-4f2e-8a68-dc3c49b71483","Type":"ContainerDied","Data":"72f1354627a6c97ed46b02384e652cf59cf549ca1d81a3554c7b4938dcdaedad"}
Jul 17 06:29:13.790181 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:13.790160    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt"
Jul 17 06:29:13.790181 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:13.790192    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72f1354627a6c97ed46b02384e652cf59cf549ca1d81a3554c7b4938dcdaedad"
Jul 17 06:29:15.752453 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:15.752422    2559 scope.go:117] "RemoveContainer" containerID="8c084c0eca830c9a157778ea5411c1aade1dd94c826bcbdac44a1a26d3d2a4b4"
Jul 17 06:29:16.807052 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:16.807014    2559 generic.go:356] "Generic (PLEG): container finished" podID="977756a1-e940-40ed-8cf9-c41e8b598867" containerID="762189d2fdf053cd87d264349b7354547527e801c232c9b38baac1105ebf57ca" exitCode=6
Jul 17 06:29:16.807547 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:16.807096    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" event={"ID":"977756a1-e940-40ed-8cf9-c41e8b598867","Type":"ContainerDied","Data":"762189d2fdf053cd87d264349b7354547527e801c232c9b38baac1105ebf57ca"}
Jul 17 06:29:16.807547 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:16.807158    2559 scope.go:117] "RemoveContainer" containerID="8c084c0eca830c9a157778ea5411c1aade1dd94c826bcbdac44a1a26d3d2a4b4"
Jul 17 06:29:16.807547 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:16.807454    2559 scope.go:117] "RemoveContainer" containerID="762189d2fdf053cd87d264349b7354547527e801c232c9b38baac1105ebf57ca"
Jul 17 06:29:16.807737 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:16.807717    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87_odh-ai-gateway-infra(977756a1-e940-40ed-8cf9-c41e8b598867)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" podUID="977756a1-e940-40ed-8cf9-c41e8b598867"
Jul 17 06:29:17.834808 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:17.834774    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"]
Jul 17 06:29:17.972061 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:17.972036    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:29:18.031122 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.031085    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977756a1-e940-40ed-8cf9-c41e8b598867-maas-api-service-ca\") pod \"977756a1-e940-40ed-8cf9-c41e8b598867\" (UID: \"977756a1-e940-40ed-8cf9-c41e8b598867\") "
Jul 17 06:29:18.031508 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.031485    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/977756a1-e940-40ed-8cf9-c41e8b598867-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "977756a1-e940-40ed-8cf9-c41e8b598867" (UID: "977756a1-e940-40ed-8cf9-c41e8b598867"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:29:18.132130 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.132036    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977756a1-e940-40ed-8cf9-c41e8b598867-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:29:18.746456 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.746421    2559 scope.go:117] "RemoveContainer" containerID="b3158bb50f70ac97fcbe2a7fa59329887dd836ca626787a1a0f42f31104d7fa1"
Jul 17 06:29:18.819448 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.819416    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87" event={"ID":"977756a1-e940-40ed-8cf9-c41e8b598867","Type":"ContainerDied","Data":"83b92bedee691ba86c32fa7da9d505991015180405219b4c39a4bef140be49fb"}
Jul 17 06:29:18.819448 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.819443    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"
Jul 17 06:29:18.819687 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.819475    2559 scope.go:117] "RemoveContainer" containerID="762189d2fdf053cd87d264349b7354547527e801c232c9b38baac1105ebf57ca"
Jul 17 06:29:18.845980 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.843841    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"]
Jul 17 06:29:18.848924 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:18.848892    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-4rs87"]
Jul 17 06:29:19.751835 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:19.751753    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" path="/var/lib/kubelet/pods/977756a1-e940-40ed-8cf9-c41e8b598867/volumes"
Jul 17 06:29:19.827102 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:19.827067    2559 generic.go:356] "Generic (PLEG): container finished" podID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerID="5be6bd6d6ad647c47f0c2db2cf36f7a1998642e4376e658bdc8aab1a943b37b8" exitCode=6
Jul 17 06:29:19.827247 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:19.827141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" event={"ID":"da6b6ee9-a785-4971-8c23-90083167d9b5","Type":"ContainerDied","Data":"5be6bd6d6ad647c47f0c2db2cf36f7a1998642e4376e658bdc8aab1a943b37b8"}
Jul 17 06:29:19.827247 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:19.827198    2559 scope.go:117] "RemoveContainer" containerID="b3158bb50f70ac97fcbe2a7fa59329887dd836ca626787a1a0f42f31104d7fa1"
Jul 17 06:29:19.827525 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:19.827493    2559 scope.go:117] "RemoveContainer" containerID="5be6bd6d6ad647c47f0c2db2cf36f7a1998642e4376e658bdc8aab1a943b37b8"
Jul 17 06:29:19.827744 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:19.827721    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4_odh-ai-gateway-infra(da6b6ee9-a785-4971-8c23-90083167d9b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5"
Jul 17 06:29:20.688636 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:20.688598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:20.853896 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:20.853843    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"]
Jul 17 06:29:20.987992 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:20.987968    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:21.061380 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.061342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da6b6ee9-a785-4971-8c23-90083167d9b5-maas-api-service-ca\") pod \"da6b6ee9-a785-4971-8c23-90083167d9b5\" (UID: \"da6b6ee9-a785-4971-8c23-90083167d9b5\") "
Jul 17 06:29:21.061695 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.061672    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da6b6ee9-a785-4971-8c23-90083167d9b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da6b6ee9-a785-4971-8c23-90083167d9b5" (UID: "da6b6ee9-a785-4971-8c23-90083167d9b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:29:21.162253 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.162211    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da6b6ee9-a785-4971-8c23-90083167d9b5-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:29:21.838604 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.838572    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"
Jul 17 06:29:21.839116 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.838604    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4" event={"ID":"da6b6ee9-a785-4971-8c23-90083167d9b5","Type":"ContainerDied","Data":"04a0a4756dda3b92b22769c70b7de9bf5507a37348ce6c6773e29c122b25c9b4"}
Jul 17 06:29:21.839116 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.838675    2559 scope.go:117] "RemoveContainer" containerID="5be6bd6d6ad647c47f0c2db2cf36f7a1998642e4376e658bdc8aab1a943b37b8"
Jul 17 06:29:21.856181 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.856149    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"]
Jul 17 06:29:21.860282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:21.860252    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-fhnb4"]
Jul 17 06:29:23.752037 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:23.752002    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" path="/var/lib/kubelet/pods/da6b6ee9-a785-4971-8c23-90083167d9b5/volumes"
Jul 17 06:29:25.185677 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:25.185636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:31.291569 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:31.291539    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:31.392759 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:31.392719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:35.863811 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:35.863779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:36.086453 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:36.086422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:42.252313 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:42.252278    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7994cc7dcc-qp6wn_3b4f78b5-d0c5-4378-a649-fbae92327340/manager/0.log"
Jul 17 06:29:42.862515 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:42.862487    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d7869b5-vqhc9_4cae1386-805f-4d01-b349-f7fa0f25e25c/copy-manifests/0.log"
Jul 17 06:29:42.874948 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:42.874919    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d7869b5-vqhc9_4cae1386-805f-4d01-b349-f7fa0f25e25c/manager/0.log"
Jul 17 06:29:43.956180 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:43.956152    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-6c5dba-x48cm_2eba1339-1ddb-4775-a96c-38f0019e0dcb/revoke-keys/0.log"
Jul 17 06:29:44.525979 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:44.525952    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-e9bcaa-7zhzz_c8c9ca9b-504b-48d8-849f-61fc762b2146/revoke-keys/0.log"
Jul 17 06:29:44.642363 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:44.642336    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-efbd92-jdtjt_4ac47d58-c5c3-4f2e-8a68-dc3c49b71483/revoke-keys/0.log"
Jul 17 06:29:46.459189 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:46.459158    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vw58v_1ddf00f8-ce20-435e-a016-9ed114c4fb85/kuadrant-console-plugin/0.log"
Jul 17 06:29:46.716728 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:46.716695    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hwwlt_9623e809-a106-495e-99d9-818388e7c314/manager/0.log"
Jul 17 06:29:47.685376 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:47.685298    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f645db4-kbsm9_d21763a2-6561-4174-b85e-0a6c36d3f1c8/kube-auth-proxy/0.log"
Jul 17 06:29:48.641831 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:48.641804    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qkq6h_40f05f67-8bcb-4cb3-a920-2587f24d324e/main/0.log"
Jul 17 06:29:48.750292 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:48.750262    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hzk6j_2931593b-a567-42fa-a7b9-69f679231d8f/main/0.log"
Jul 17 06:29:48.977799 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:48.977764    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-j5mvz_fb2f5051-ea0a-4f4f-98ed-a63cf959a4e2/main/0.log"
Jul 17 06:29:53.168660 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.168625    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"]
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169012    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169026    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169033    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169038    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169052    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169058    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169064    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169070    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169080    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ac47d58-c5c3-4f2e-8a68-dc3c49b71483" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169086    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ac47d58-c5c3-4f2e-8a68-dc3c49b71483" containerName="revoke-keys"
Jul 17 06:29:53.169086 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169092    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169097    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169111    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169116    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169167    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169176    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169182    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169189    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6b6ee9-a785-4971-8c23-90083167d9b5" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169196    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ac47d58-c5c3-4f2e-8a68-dc3c49b71483" containerName="revoke-keys"
Jul 17 06:29:53.169431 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.169201    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:53.172294 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.172274    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:29:53.175184 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.175164    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 06:29:53.175311 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.175164    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pcjfr\""
Jul 17 06:29:53.179073 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.179045    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"]
Jul 17 06:29:53.249975 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.249939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac6e74-16c9-455b-afd6-484b7b9992cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz\" (UID: \"a4ac6e74-16c9-455b-afd6-484b7b9992cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:29:53.350925 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.350849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac6e74-16c9-455b-afd6-484b7b9992cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz\" (UID: \"a4ac6e74-16c9-455b-afd6-484b7b9992cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:29:53.351513 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.351493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac6e74-16c9-455b-afd6-484b7b9992cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz\" (UID: \"a4ac6e74-16c9-455b-afd6-484b7b9992cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:29:53.484279 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.484230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:29:53.623852 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.623827    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"]
Jul 17 06:29:53.626490 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:29:53.626456    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ac6e74_16c9_455b_afd6_484b7b9992cf.slice/crio-8ed07823ee596b58520a8fc357cb0fa5200f91fd666cce31bf06fc8b6a03179d WatchSource:0}: Error finding container 8ed07823ee596b58520a8fc357cb0fa5200f91fd666cce31bf06fc8b6a03179d: Status 404 returned error can't find the container with id 8ed07823ee596b58520a8fc357cb0fa5200f91fd666cce31bf06fc8b6a03179d
Jul 17 06:29:53.996231 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.996191    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerStarted","Data":"001f08e8468e9c281652768c123bcdc8525b7fbb1a7275344d7a88655ca242e6"}
Jul 17 06:29:53.996404 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:53.996259    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerStarted","Data":"8ed07823ee596b58520a8fc357cb0fa5200f91fd666cce31bf06fc8b6a03179d"}
Jul 17 06:29:54.014449 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:54.014389    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" podStartSLOduration=1.014369932 podStartE2EDuration="1.014369932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:29:54.013457587 +0000 UTC m=+3418.791940551" watchObservedRunningTime="2026-07-17 06:29:54.014369932 +0000 UTC m=+3418.792852897"
Jul 17 06:29:54.487517 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:54.487479    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:54.587611 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:54.587526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:55.003797 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:55.003759    2559 generic.go:356] "Generic (PLEG): container finished" podID="a4ac6e74-16c9-455b-afd6-484b7b9992cf" containerID="001f08e8468e9c281652768c123bcdc8525b7fbb1a7275344d7a88655ca242e6" exitCode=6
Jul 17 06:29:55.004032 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:55.003841    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerDied","Data":"001f08e8468e9c281652768c123bcdc8525b7fbb1a7275344d7a88655ca242e6"}
Jul 17 06:29:55.004137 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:55.004103    2559 scope.go:117] "RemoveContainer" containerID="001f08e8468e9c281652768c123bcdc8525b7fbb1a7275344d7a88655ca242e6"
Jul 17 06:29:55.737734 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:55.737698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:55.841203 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:55.841120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log" limit=1024
Jul 17 06:29:56.011585 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:56.011537    2559 generic.go:356] "Generic (PLEG): container finished" podID="a4ac6e74-16c9-455b-afd6-484b7b9992cf" containerID="2bd3b31070ebbe2fe9a93adc00533fe89ceed370cac2849a3118d7ecb287164e" exitCode=6
Jul 17 06:29:56.011777 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:56.011623    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerDied","Data":"2bd3b31070ebbe2fe9a93adc00533fe89ceed370cac2849a3118d7ecb287164e"}
Jul 17 06:29:56.011777 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:56.011684    2559 scope.go:117] "RemoveContainer" containerID="001f08e8468e9c281652768c123bcdc8525b7fbb1a7275344d7a88655ca242e6"
Jul 17 06:29:56.011991 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:56.011973    2559 scope.go:117] "RemoveContainer" containerID="2bd3b31070ebbe2fe9a93adc00533fe89ceed370cac2849a3118d7ecb287164e"
Jul 17 06:29:56.012228 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:56.012208    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz_odh-ai-gateway-infra(a4ac6e74-16c9-455b-afd6-484b7b9992cf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" podUID="a4ac6e74-16c9-455b-afd6-484b7b9992cf"
Jul 17 06:29:57.021489 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:57.021461    2559 scope.go:117] "RemoveContainer" containerID="2bd3b31070ebbe2fe9a93adc00533fe89ceed370cac2849a3118d7ecb287164e"
Jul 17 06:29:57.021896 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:29:57.021682    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz_odh-ai-gateway-infra(a4ac6e74-16c9-455b-afd6-484b7b9992cf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" podUID="a4ac6e74-16c9-455b-afd6-484b7b9992cf"
Jul 17 06:29:57.312882 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:57.312777    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7gz6h_93a50d18-6fa3-4037-8174-4ff5a09f6fb8/global-pull-secret-syncer/0.log"
Jul 17 06:29:57.439387 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:57.439350    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mhjrt_fdb15470-3a00-4dc9-9b56-ba56e4661ab1/konnectivity-agent/0.log"
Jul 17 06:29:57.542287 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:57.542253    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-189.ec2.internal_306dc5b5dbae4709c425668aaebe9719/haproxy/0.log"
Jul 17 06:29:58.837777 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:58.837736    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"]
Jul 17 06:29:58.838216 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:58.838173    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="977756a1-e940-40ed-8cf9-c41e8b598867" containerName="revoke-keys"
Jul 17 06:29:58.842968 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:58.842951    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:29:58.849879 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:58.849827    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"]
Jul 17 06:29:58.898159 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:58.898119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/648d1680-7262-420c-b7ed-b25ebf6b9d9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6\" (UID: \"648d1680-7262-420c-b7ed-b25ebf6b9d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:29:58.999338 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:58.999298    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/648d1680-7262-420c-b7ed-b25ebf6b9d9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6\" (UID: \"648d1680-7262-420c-b7ed-b25ebf6b9d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:29:59.000030 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:59.000011    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/648d1680-7262-420c-b7ed-b25ebf6b9d9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6\" (UID: \"648d1680-7262-420c-b7ed-b25ebf6b9d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:29:59.155888 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:59.155749    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:29:59.294865 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:29:59.294828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"]
Jul 17 06:29:59.297258 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:29:59.297219    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648d1680_7262_420c_b7ed_b25ebf6b9d9b.slice/crio-e46352ae6885d9a68d6f76f17c5a388e1303ba38ad0c7c8a549bbdfbadd1843d WatchSource:0}: Error finding container e46352ae6885d9a68d6f76f17c5a388e1303ba38ad0c7c8a549bbdfbadd1843d: Status 404 returned error can't find the container with id e46352ae6885d9a68d6f76f17c5a388e1303ba38ad0c7c8a549bbdfbadd1843d
Jul 17 06:30:00.037714 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:00.037683    2559 generic.go:356] "Generic (PLEG): container finished" podID="648d1680-7262-420c-b7ed-b25ebf6b9d9b" containerID="9952ad5e104cc57fad241084b2fd2e503a002b1ffdf5271b6a38cd6589ba943f" exitCode=6
Jul 17 06:30:00.038139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:00.037778    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" event={"ID":"648d1680-7262-420c-b7ed-b25ebf6b9d9b","Type":"ContainerDied","Data":"9952ad5e104cc57fad241084b2fd2e503a002b1ffdf5271b6a38cd6589ba943f"}
Jul 17 06:30:00.038139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:00.037832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" event={"ID":"648d1680-7262-420c-b7ed-b25ebf6b9d9b","Type":"ContainerStarted","Data":"e46352ae6885d9a68d6f76f17c5a388e1303ba38ad0c7c8a549bbdfbadd1843d"}
Jul 17 06:30:00.038139 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:00.037947    2559 scope.go:117] "RemoveContainer" containerID="9952ad5e104cc57fad241084b2fd2e503a002b1ffdf5271b6a38cd6589ba943f"
Jul 17 06:30:01.044567 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:01.044530    2559 generic.go:356] "Generic (PLEG): container finished" podID="648d1680-7262-420c-b7ed-b25ebf6b9d9b" containerID="3c6886f4d63842c5687d06af8fd96231c70883eefcf0f167f3f6bbf4f778710f" exitCode=6
Jul 17 06:30:01.045078 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:01.044615    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" event={"ID":"648d1680-7262-420c-b7ed-b25ebf6b9d9b","Type":"ContainerDied","Data":"3c6886f4d63842c5687d06af8fd96231c70883eefcf0f167f3f6bbf4f778710f"}
Jul 17 06:30:01.045078 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:01.044680    2559 scope.go:117] "RemoveContainer" containerID="9952ad5e104cc57fad241084b2fd2e503a002b1ffdf5271b6a38cd6589ba943f"
Jul 17 06:30:01.045078 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:01.044909    2559 scope.go:117] "RemoveContainer" containerID="3c6886f4d63842c5687d06af8fd96231c70883eefcf0f167f3f6bbf4f778710f"
Jul 17 06:30:01.045252 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:30:01.045133    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6_odh-ai-gateway-infra(648d1680-7262-420c-b7ed-b25ebf6b9d9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" podUID="648d1680-7262-420c-b7ed-b25ebf6b9d9b"
Jul 17 06:30:02.056352 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:02.056322    2559 scope.go:117] "RemoveContainer" containerID="3c6886f4d63842c5687d06af8fd96231c70883eefcf0f167f3f6bbf4f778710f"
Jul 17 06:30:02.056765 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:30:02.056550    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6_odh-ai-gateway-infra(648d1680-7262-420c-b7ed-b25ebf6b9d9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" podUID="648d1680-7262-420c-b7ed-b25ebf6b9d9b"
Jul 17 06:30:02.450111 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:02.450014    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vw58v_1ddf00f8-ce20-435e-a016-9ed114c4fb85/kuadrant-console-plugin/0.log"
Jul 17 06:30:02.579707 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:02.579669    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hwwlt_9623e809-a106-495e-99d9-818388e7c314/manager/0.log"
Jul 17 06:30:04.504770 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:04.504736    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-74mbk_a1df72a6-3180-44b4-a30d-a3e1917adccb/cluster-monitoring-operator/0.log"
Jul 17 06:30:04.649436 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:04.649405    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-kn4z7_49130648-2b3a-4e3b-afb2-934db8e9b287/monitoring-plugin/0.log"
Jul 17 06:30:04.681840 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:04.681802    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-glwxm_f58dc1c9-0eb2-4631-8813-21fa11f0cf63/node-exporter/0.log"
Jul 17 06:30:04.701510 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:04.701483    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-glwxm_f58dc1c9-0eb2-4631-8813-21fa11f0cf63/kube-rbac-proxy/0.log"
Jul 17 06:30:04.721183 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:04.721153    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-glwxm_f58dc1c9-0eb2-4631-8813-21fa11f0cf63/init-textfile/0.log"
Jul 17 06:30:05.081250 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.081220    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/prometheus/0.log"
Jul 17 06:30:05.101541 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.101432    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/config-reloader/0.log"
Jul 17 06:30:05.119795 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.119748    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/thanos-sidecar/0.log"
Jul 17 06:30:05.141308 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.141278    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/kube-rbac-proxy-web/0.log"
Jul 17 06:30:05.163626 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.163597    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/kube-rbac-proxy/0.log"
Jul 17 06:30:05.185009 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.184981    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/kube-rbac-proxy-thanos/0.log"
Jul 17 06:30:05.213653 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.213626    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7b86a9f-605a-4397-b4f5-db298d1200f8/init-config-reloader/0.log"
Jul 17 06:30:05.242275 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.242246    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-b7l6l_9b79b45e-5bcb-43ec-b855-a2e6168620d5/prometheus-operator/0.log"
Jul 17 06:30:05.260647 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.260622    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-b7l6l_9b79b45e-5bcb-43ec-b855-a2e6168620d5/kube-rbac-proxy/0.log"
Jul 17 06:30:05.295265 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:05.295230    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-rvj4z_a960fc61-ed51-42d3-8478-b0e5fbe27dd3/prometheus-operator-admission-webhook/0.log"
Jul 17 06:30:08.084185 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.084149    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"]
Jul 17 06:30:08.088907 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.088882    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.091903 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.091880    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n6jcf\"/\"openshift-service-ca.crt\""
Jul 17 06:30:08.091903 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.091885    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n6jcf\"/\"kube-root-ca.crt\""
Jul 17 06:30:08.092102 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.091903    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n6jcf\"/\"default-dockercfg-jcjq2\""
Jul 17 06:30:08.096516 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.096492    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"]
Jul 17 06:30:08.181316 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.181276    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tgfm\" (UniqueName: \"kubernetes.io/projected/c1751138-9d12-41a0-b72d-b8707708f854-kube-api-access-2tgfm\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.181316 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.181322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-proc\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.181549 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.181341    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-lib-modules\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.181549 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.181374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-podres\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.181549 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.181427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-sys\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282263 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-proc\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282263 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-lib-modules\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-proc\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-podres\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282436    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-sys\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282503 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282476    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-podres\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282663 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-sys\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282663 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1751138-9d12-41a0-b72d-b8707708f854-lib-modules\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.282663 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.282574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tgfm\" (UniqueName: \"kubernetes.io/projected/c1751138-9d12-41a0-b72d-b8707708f854-kube-api-access-2tgfm\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.291373 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.291336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tgfm\" (UniqueName: \"kubernetes.io/projected/c1751138-9d12-41a0-b72d-b8707708f854-kube-api-access-2tgfm\") pod \"perf-node-gather-daemonset-6sjnp\" (UID: \"c1751138-9d12-41a0-b72d-b8707708f854\") " pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.399971 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.399840    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:08.541104 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.541074    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"]
Jul 17 06:30:08.543260 ip-10-0-128-189 kubenswrapper[2559]: W0717 06:30:08.543233    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc1751138_9d12_41a0_b72d_b8707708f854.slice/crio-ac52e402072f00bcb5e5917dfd032ea610d4f5d4ddcad82ea9eb40b7737f3566 WatchSource:0}: Error finding container ac52e402072f00bcb5e5917dfd032ea610d4f5d4ddcad82ea9eb40b7737f3566: Status 404 returned error can't find the container with id ac52e402072f00bcb5e5917dfd032ea610d4f5d4ddcad82ea9eb40b7737f3566
Jul 17 06:30:08.710596 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:08.710565    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-7bwdw_7f766e54-ad49-4a16-a8a2-71e727dbeab2/volume-data-source-validator/0.log"
Jul 17 06:30:09.091175 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.091140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp" event={"ID":"c1751138-9d12-41a0-b72d-b8707708f854","Type":"ContainerStarted","Data":"da92ad58774ec8c48970fb4dd008cd0350ab502823e35da03c955e2ab91a55c6"}
Jul 17 06:30:09.091595 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.091198    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:09.091595 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.091218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp" event={"ID":"c1751138-9d12-41a0-b72d-b8707708f854","Type":"ContainerStarted","Data":"ac52e402072f00bcb5e5917dfd032ea610d4f5d4ddcad82ea9eb40b7737f3566"}
Jul 17 06:30:09.107799 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.107746    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp" podStartSLOduration=1.10773021 podStartE2EDuration="1.10773021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 06:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:30:09.106309008 +0000 UTC m=+3433.884791968" watchObservedRunningTime="2026-07-17 06:30:09.10773021 +0000 UTC m=+3433.886213170"
Jul 17 06:30:09.728755 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.728721    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m5m5_53c331fb-d346-4c18-b9eb-d0b8a8825437/dns/0.log"
Jul 17 06:30:09.745929 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.745899    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m5m5_53c331fb-d346-4c18-b9eb-d0b8a8825437/kube-rbac-proxy/0.log"
Jul 17 06:30:09.746104 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.746034    2559 scope.go:117] "RemoveContainer" containerID="2bd3b31070ebbe2fe9a93adc00533fe89ceed370cac2849a3118d7ecb287164e"
Jul 17 06:30:09.808108 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:09.808066    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4mb5k_b8649966-7fcf-46ad-af87-392af43bb3df/dns-node-resolver/0.log"
Jul 17 06:30:10.097593 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:10.097560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerStarted","Data":"c0bf9d32d4bbc8c8aab00d938df71f9144ba343d422a28d5c5f816ed0f8fae81"}
Jul 17 06:30:10.385452 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:10.385370    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-47vtj_0833fd9c-124f-4b64-9618-cac8678daad0/node-ca/0.log"
Jul 17 06:30:11.103765 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:11.103726    2559 generic.go:356] "Generic (PLEG): container finished" podID="a4ac6e74-16c9-455b-afd6-484b7b9992cf" containerID="c0bf9d32d4bbc8c8aab00d938df71f9144ba343d422a28d5c5f816ed0f8fae81" exitCode=6
Jul 17 06:30:11.104282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:11.103799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerDied","Data":"c0bf9d32d4bbc8c8aab00d938df71f9144ba343d422a28d5c5f816ed0f8fae81"}
Jul 17 06:30:11.104282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:11.103889    2559 scope.go:117] "RemoveContainer" containerID="2bd3b31070ebbe2fe9a93adc00533fe89ceed370cac2849a3118d7ecb287164e"
Jul 17 06:30:11.104282 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:11.104130    2559 scope.go:117] "RemoveContainer" containerID="c0bf9d32d4bbc8c8aab00d938df71f9144ba343d422a28d5c5f816ed0f8fae81"
Jul 17 06:30:11.104461 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:30:11.104357    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz_odh-ai-gateway-infra(a4ac6e74-16c9-455b-afd6-484b7b9992cf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" podUID="a4ac6e74-16c9-455b-afd6-484b7b9992cf"
Jul 17 06:30:11.126712 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:11.126676    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"]
Jul 17 06:30:11.470265 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:11.470238    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f645db4-kbsm9_d21763a2-6561-4174-b85e-0a6c36d3f1c8/kube-auth-proxy/0.log"
Jul 17 06:30:12.252439 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.252414    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:30:12.319725 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.319691    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac6e74-16c9-455b-afd6-484b7b9992cf-maas-api-service-ca\") pod \"a4ac6e74-16c9-455b-afd6-484b7b9992cf\" (UID: \"a4ac6e74-16c9-455b-afd6-484b7b9992cf\") "
Jul 17 06:30:12.320126 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.320103    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ac6e74-16c9-455b-afd6-484b7b9992cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4ac6e74-16c9-455b-afd6-484b7b9992cf" (UID: "a4ac6e74-16c9-455b-afd6-484b7b9992cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:30:12.338691 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.338660    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzrw2_d4b17b3a-cc6c-47fe-b73e-c7a6966e34ae/serve-healthcheck-canary/0.log"
Jul 17 06:30:12.420989 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.420884    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac6e74-16c9-455b-afd6-484b7b9992cf-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:30:12.833660 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.833562    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-czqrr_34afd671-dce9-4fce-8fa0-c265ca199e21/insights-operator/0.log"
Jul 17 06:30:12.834467 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.834450    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-czqrr_34afd671-dce9-4fce-8fa0-c265ca199e21/insights-operator/1.log"
Jul 17 06:30:12.990070 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:12.990039    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x6h6s_2d502e8a-66f0-460d-9b74-b86338a4e52a/kube-rbac-proxy/0.log"
Jul 17 06:30:13.009144 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.009117    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x6h6s_2d502e8a-66f0-460d-9b74-b86338a4e52a/exporter/0.log"
Jul 17 06:30:13.029666 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.029638    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x6h6s_2d502e8a-66f0-460d-9b74-b86338a4e52a/extractor/0.log"
Jul 17 06:30:13.115912 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.115804    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"
Jul 17 06:30:13.115912 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.115873    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz" event={"ID":"a4ac6e74-16c9-455b-afd6-484b7b9992cf","Type":"ContainerDied","Data":"8ed07823ee596b58520a8fc357cb0fa5200f91fd666cce31bf06fc8b6a03179d"}
Jul 17 06:30:13.116174 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.115942    2559 scope.go:117] "RemoveContainer" containerID="c0bf9d32d4bbc8c8aab00d938df71f9144ba343d422a28d5c5f816ed0f8fae81"
Jul 17 06:30:13.140276 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.140239    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"]
Jul 17 06:30:13.145939 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.145911    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-583f1136-d4pxz"]
Jul 17 06:30:13.752193 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:13.752156    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ac6e74-16c9-455b-afd6-484b7b9992cf" path="/var/lib/kubelet/pods/a4ac6e74-16c9-455b-afd6-484b7b9992cf/volumes"
Jul 17 06:30:15.107766 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:15.107729    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n6jcf/perf-node-gather-daemonset-6sjnp"
Jul 17 06:30:15.551611 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:15.551566    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7994cc7dcc-qp6wn_3b4f78b5-d0c5-4378-a649-fbae92327340/manager/0.log"
Jul 17 06:30:15.835998 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:15.835907    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d7869b5-vqhc9_4cae1386-805f-4d01-b349-f7fa0f25e25c/manager/0.log"
Jul 17 06:30:15.854592 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:15.854567    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d7869b5-vqhc9_4cae1386-805f-4d01-b349-f7fa0f25e25c/copy-manifests/0.log"
Jul 17 06:30:17.181253 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:17.181220    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6687b4bb84-4dsgv_523a88cd-80d2-4a3c-931a-84e88a0a805c/manager/0.log"
Jul 17 06:30:17.233975 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:17.233938    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rxlfw_ad56b8ef-f9e4-452c-9209-41ba236254c3/openshift-lws-operator/0.log"
Jul 17 06:30:17.746536 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:17.746509    2559 scope.go:117] "RemoveContainer" containerID="3c6886f4d63842c5687d06af8fd96231c70883eefcf0f167f3f6bbf4f778710f"
Jul 17 06:30:17.935469 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:17.935427    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f645db4-kbsm9_d21763a2-6561-4174-b85e-0a6c36d3f1c8/kube-auth-proxy/0.log"
Jul 17 06:30:18.144100 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:18.144057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" event={"ID":"648d1680-7262-420c-b7ed-b25ebf6b9d9b","Type":"ContainerStarted","Data":"deecccaabfec98e7dcb1a00a5a3e90bcb0b169a001834c3b13aab4472a07be1f"}
Jul 17 06:30:18.163006 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:18.162942    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" podStartSLOduration=20.162923601 podStartE2EDuration="20.162923601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:30:18.161432951 +0000 UTC m=+3442.939915912" watchObservedRunningTime="2026-07-17 06:30:18.162923601 +0000 UTC m=+3442.941406561"
Jul 17 06:30:19.150733 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:19.150698    2559 generic.go:356] "Generic (PLEG): container finished" podID="648d1680-7262-420c-b7ed-b25ebf6b9d9b" containerID="deecccaabfec98e7dcb1a00a5a3e90bcb0b169a001834c3b13aab4472a07be1f" exitCode=6
Jul 17 06:30:19.151209 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:19.150767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" event={"ID":"648d1680-7262-420c-b7ed-b25ebf6b9d9b","Type":"ContainerDied","Data":"deecccaabfec98e7dcb1a00a5a3e90bcb0b169a001834c3b13aab4472a07be1f"}
Jul 17 06:30:19.151209 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:19.150829    2559 scope.go:117] "RemoveContainer" containerID="3c6886f4d63842c5687d06af8fd96231c70883eefcf0f167f3f6bbf4f778710f"
Jul 17 06:30:19.151209 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:19.151092    2559 scope.go:117] "RemoveContainer" containerID="deecccaabfec98e7dcb1a00a5a3e90bcb0b169a001834c3b13aab4472a07be1f"
Jul 17 06:30:19.151368 ip-10-0-128-189 kubenswrapper[2559]: E0717 06:30:19.151346    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6_odh-ai-gateway-infra(648d1680-7262-420c-b7ed-b25ebf6b9d9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" podUID="648d1680-7262-420c-b7ed-b25ebf6b9d9b"
Jul 17 06:30:19.176871 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:19.176831    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"]
Jul 17 06:30:19.395395 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:19.395287    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f645db4-kbsm9_d21763a2-6561-4174-b85e-0a6c36d3f1c8/kube-auth-proxy/0.log"
Jul 17 06:30:20.296568 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:20.296541    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:30:20.393643 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:20.393601    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/648d1680-7262-420c-b7ed-b25ebf6b9d9b-maas-api-service-ca\") pod \"648d1680-7262-420c-b7ed-b25ebf6b9d9b\" (UID: \"648d1680-7262-420c-b7ed-b25ebf6b9d9b\") "
Jul 17 06:30:20.394081 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:20.394058    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/648d1680-7262-420c-b7ed-b25ebf6b9d9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "648d1680-7262-420c-b7ed-b25ebf6b9d9b" (UID: "648d1680-7262-420c-b7ed-b25ebf6b9d9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:30:20.494954 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:20.494913    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/648d1680-7262-420c-b7ed-b25ebf6b9d9b-maas-api-service-ca\") on node \"ip-10-0-128-189.ec2.internal\" DevicePath \"\""
Jul 17 06:30:21.164364 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:21.164323    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"
Jul 17 06:30:21.164364 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:21.164344    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6" event={"ID":"648d1680-7262-420c-b7ed-b25ebf6b9d9b","Type":"ContainerDied","Data":"e46352ae6885d9a68d6f76f17c5a388e1303ba38ad0c7c8a549bbdfbadd1843d"}
Jul 17 06:30:21.164686 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:21.164411    2559 scope.go:117] "RemoveContainer" containerID="deecccaabfec98e7dcb1a00a5a3e90bcb0b169a001834c3b13aab4472a07be1f"
Jul 17 06:30:21.191490 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:21.191455    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"]
Jul 17 06:30:21.198114 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:21.198085    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-97f0d5b3-wcqh6"]
Jul 17 06:30:21.752246 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:21.752212    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="648d1680-7262-420c-b7ed-b25ebf6b9d9b" path="/var/lib/kubelet/pods/648d1680-7262-420c-b7ed-b25ebf6b9d9b/volumes"
Jul 17 06:30:23.640334 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.640305    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/kube-multus-additional-cni-plugins/0.log"
Jul 17 06:30:23.658453 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.658423    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/egress-router-binary-copy/0.log"
Jul 17 06:30:23.676010 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.675976    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/cni-plugins/0.log"
Jul 17 06:30:23.693831 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.693803    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/bond-cni-plugin/0.log"
Jul 17 06:30:23.713071 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.713042    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/routeoverride-cni/0.log"
Jul 17 06:30:23.730187 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.730161    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/whereabouts-cni-bincopy/0.log"
Jul 17 06:30:23.748109 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:23.748075    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c2f5t_3c70031c-daa9-4b4e-a4a4-0c52ac2d0354/whereabouts-cni/0.log"
Jul 17 06:30:24.089042 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:24.088995    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fb5fx_c7b84b7a-c01d-4f80-8605-b8624b272347/kube-multus/0.log"
Jul 17 06:30:24.283732 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:24.283692    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rk2tl_6a08ba58-2146-46da-bb66-1132a32b2419/network-metrics-daemon/0.log"
Jul 17 06:30:24.302977 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:24.302946    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rk2tl_6a08ba58-2146-46da-bb66-1132a32b2419/kube-rbac-proxy/0.log"
Jul 17 06:30:25.167212 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.167179    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/ovn-controller/0.log"
Jul 17 06:30:25.188693 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.188662    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/ovn-acl-logging/0.log"
Jul 17 06:30:25.216519 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.216487    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/kube-rbac-proxy-node/0.log"
Jul 17 06:30:25.240691 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.240660    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/kube-rbac-proxy-ovn-metrics/0.log"
Jul 17 06:30:25.261920 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.261891    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/northd/0.log"
Jul 17 06:30:25.283695 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.283669    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/nbdb/0.log"
Jul 17 06:30:25.307243 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.307216    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/sbdb/0.log"
Jul 17 06:30:25.388508 ip-10-0-128-189 kubenswrapper[2559]: I0717 06:30:25.388477    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4l8jz_2aa96119-a12d-460d-9b53-d2c15e27bc59/ovnkube-controller/0.log"