Aug 04 11:33:06.301547 ip-10-0-129-11 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 11:33:06.748469 ip-10-0-129-11 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 11:33:06.748469 ip-10-0-129-11 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 11:33:06.748469 ip-10-0-129-11 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 11:33:06.748469 ip-10-0-129-11 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 11:33:06.748469 ip-10-0-129-11 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 11:33:06.750154 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.749993    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 11:33:06.756417 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756394    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 11:33:06.756417 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756413    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 11:33:06.756417 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756419    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 11:33:06.756417 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756424    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756429    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756434    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756439    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756443    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756447    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756451    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756455    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756460    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756464    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756468    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756473    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756477    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756481    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756485    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756490    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756494    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756506    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756511    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756517    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 11:33:06.756674 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756521    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756525    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756530    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756535    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756540    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756545    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756549    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756554    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756559    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756564    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756568    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756573    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756577    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756582    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756588    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756603    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756608    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756612    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756617    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756621    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 11:33:06.757590 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756626    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756631    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756635    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756640    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756645    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756649    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756654    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756659    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756664    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756669    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756673    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756677    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756681    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756686    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756691    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756696    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756700    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756705    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756709    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756714    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 11:33:06.758569 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756719    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756724    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756728    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756733    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756738    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756743    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756749    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756754    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756760    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756765    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756770    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756775    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756779    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756784    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756791    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756796    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756800    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756805    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756814    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 11:33:06.759459 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756821    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756826    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756831    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756836    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756841    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756845    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756850    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756855    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756862    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756868    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756874    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756879    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756886    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756893    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756898    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756902    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.756907    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758161    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758171    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758178    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 11:33:06.760370 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758183    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758188    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758194    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758199    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758203    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758208    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758215    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758222    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758227    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758232    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758237    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758241    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758246    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758250    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758255    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758259    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758264    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758268    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758273    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758277    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 11:33:06.761287 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758282    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758287    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758292    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758297    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758301    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758306    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758310    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758315    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758319    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758324    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758329    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758335    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758339    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758344    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758348    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758374    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758379    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758384    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758388    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758393    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 11:33:06.762073 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758398    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758403    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758407    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758412    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758417    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758421    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758426    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758430    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758435    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758440    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758444    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758449    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758453    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758458    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758463    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758468    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758473    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758479    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758484    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758489    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 11:33:06.762871 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758493    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758499    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758503    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758508    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758513    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758517    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758522    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758526    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758532    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758537    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758542    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758549    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758554    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758558    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758565    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758570    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758575    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758580    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758585    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 11:33:06.763540 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758591    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758596    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758601    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758606    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758611    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758616    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758620    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758624    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758629    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758634    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758639    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758646    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758652    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758658    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758664    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758669    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.758673    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759376    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759386    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 11:33:06.764054 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759395    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759402    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759411    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759417    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759422    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759427    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759431    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759436    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759441    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759446    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759451    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759455    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759460    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759464    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759469    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759474    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759478    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759484    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759488    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759493    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 11:33:06.764689 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759497    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759502    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759507    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759513    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759518    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759522    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759527    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759532    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759536    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759541    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759546    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759550    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759555    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759560    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759564    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759569    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759574    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759578    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759586    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 11:33:06.765340 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759592    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759597    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759602    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759607    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759612    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759617    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759624    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759629    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759634    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759639    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759643    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759648    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759652    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759657    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759661    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759666    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759670    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759675    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759679    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759684    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 11:33:06.765877 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759689    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759694    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759698    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759712    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759717    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759721    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759726    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759731    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759736    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759742    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759746    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759750    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759755    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759760    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759764    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759768    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759773    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759777    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759784    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759789    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 11:33:06.766427 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759793    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759797    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759802    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759806    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759811    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759815    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759821    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759825    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759830    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759835    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759839    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759844    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759848    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759852    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759857    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759861    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759866    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.759870    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.759995    2558 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760007    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760019    2558 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 11:33:06.766944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760027    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760034    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760041    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760049    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760057    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760063    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760068    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760074    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760080    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760087    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760093    2558 flags.go:64] FLAG: --cgroup-root=""
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760099    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760104    2558 flags.go:64] FLAG: --client-ca-file=""
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760109    2558 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760114    2558 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760121    2558 flags.go:64] FLAG: --cluster-domain=""
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760126    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760132    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760137    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760143    2558 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760151    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760156    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760162    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 11:33:06.767578 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760168    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760174    2558 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760179    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760184    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760190    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760195    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760202    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760207    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760212    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760217    2558 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760222    2558 flags.go:64] FLAG: --enable-server="true"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760228    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760238    2558 flags.go:64] FLAG: --event-burst="100"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760243    2558 flags.go:64] FLAG: --event-qps="50"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760248    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760254    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760259    2558 flags.go:64] FLAG: --eviction-hard=""
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760265    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760273    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760278    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760284    2558 flags.go:64] FLAG: --eviction-soft=""
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760288    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760293    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760299    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760304    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 11:33:06.768177 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760309    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760314    2558 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760319    2558 flags.go:64] FLAG: --feature-gates=""
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760326    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760331    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760336    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760341    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760347    2558 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760367    2558 flags.go:64] FLAG: --help="false"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760374    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-129-11.ec2.internal"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760379    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760384    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760389    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760395    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760401    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760406    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760411    2558 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760416    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760421    2558 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760428    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760434    2558 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760439    2558 flags.go:64] FLAG: --kube-reserved=""
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760444    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760450    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 11:33:06.768907 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760455    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760467    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760472    2558 flags.go:64] FLAG: --lock-file=""
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760477    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760483    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760488    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760497    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760502    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760508    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760513    2558 flags.go:64] FLAG: --logging-format="text"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760519    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760525    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760531    2558 flags.go:64] FLAG: --manifest-url=""
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760536    2558 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760543    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760549    2558 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760556    2558 flags.go:64] FLAG: --max-pods="110"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760562    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760567    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760572    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760578    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760583    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760588    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760593    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760605    2558 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 11:33:06.769571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760610    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760616    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760621    2558 flags.go:64] FLAG: --pod-cidr=""
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760627    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760638    2558 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760643    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760648    2558 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760653    2558 flags.go:64] FLAG: --port="10250"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760660    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760665    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05ba6b707421ce2be"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760671    2558 flags.go:64] FLAG: --qos-reserved=""
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760677    2558 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760682    2558 flags.go:64] FLAG: --register-node="true"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760687    2558 flags.go:64] FLAG: --register-with-taints=""
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760694    2558 flags.go:64] FLAG: --registry-burst="10"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760699    2558 flags.go:64] FLAG: --registry-qps="5"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760704    2558 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760709    2558 flags.go:64] FLAG: --reserved-memory=""
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760715    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760723    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760729    2558 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760734    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760739    2558 flags.go:64] FLAG: --runonce="false"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760745    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760750    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 11:33:06.770240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760755    2558 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760760    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760766    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760772    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760777    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760782    2558 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760787    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760793    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760797    2558 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760802    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760808    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760814    2558 flags.go:64] FLAG: --system-cgroups=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760819    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760829    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760834    2558 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760840    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760847    2558 flags.go:64] FLAG: --tls-min-version=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760852    2558 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760857    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760862    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760868    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760873    2558 flags.go:64] FLAG: --v="2"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760880    2558 flags.go:64] FLAG: --version="false"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760886    2558 flags.go:64] FLAG: --vmodule=""
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760893    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 11:33:06.770975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.760899    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761106    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761116    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761123    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761128    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761134    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761139    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761144    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761149    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761154    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761158    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761163    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761169    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761174    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761179    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761184    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761189    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761194    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761199    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761204    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 11:33:06.771654 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761209    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761214    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761221    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761225    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761229    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761234    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761238    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761242    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761247    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761252    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761256    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761261    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761268    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761276    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761281    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761286    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761290    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761294    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761299    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761304    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 11:33:06.772202 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761308    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761312    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761317    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761321    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761326    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761330    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761335    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761340    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761345    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761349    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761374    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761379    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761385    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761390    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761398    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761403    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761407    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761412    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761417    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 11:33:06.772801 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761422    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761427    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761432    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761437    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761441    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761448    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761457    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761462    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761467    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761472    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761476    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761481    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761486    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761490    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761494    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761499    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761504    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761509    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761513    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 11:33:06.773343 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761518    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761522    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761526    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761531    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761537    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761541    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761546    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761551    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761558    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761562    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761567    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761571    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761575    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761580    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761585    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761589    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761594    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761598    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761603    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761609    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 11:33:06.773878 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761613    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.761618    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.761628    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.769127    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.769144    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.769164    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.769171    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769230    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769236    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769240    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769244    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769247    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769250    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769253    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769256    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 11:33:06.774420 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769259    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769262    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769265    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769268    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769271    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769274    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769277    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769279    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769282    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769285    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769288    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769291    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769295    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769298    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769301    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769304    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769307    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769310    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769312    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769315    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 11:33:06.774833 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769319    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769323    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769327    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769331    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769335    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769338    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769341    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769344    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769347    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769350    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769369    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769372    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769375    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769378    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769381    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769384    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769387    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769390    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769393    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 11:33:06.775446 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769396    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769399    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769401    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769404    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769407    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769410    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769413    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769416    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769419    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769421    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769424    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769427    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769430    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769434    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769437    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769440    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769443    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769446    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769449    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769452    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 11:33:06.775972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769455    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769458    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769461    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769464    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769467    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769470    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769474    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769478    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769481    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769484    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769487    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769490    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769493    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769496    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769499    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769501    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769504    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769507    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769510    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 11:33:06.776538 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769512    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769515    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769518    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769521    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769524    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769526    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769530    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769533    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769536    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769540    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769543    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769546    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769549    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.769555    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 11:33:06.777100 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769680    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769685    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769689    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769693    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769696    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769699    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769702    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769705    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769708    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769711    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769713    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769716    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769719    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769722    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769725    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769727    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769730    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769733    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769736    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 11:33:06.777621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769739    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769742    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769744    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769747    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769750    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769753    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769756    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769759    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769761    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769765    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769767    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769770    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769773    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769776    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769778    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769781    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769784    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769787    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769790    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769809    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 11:33:06.778490 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769813    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769817    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769820    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769823    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769826    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769829    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769831    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769834    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769837    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769840    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769842    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769845    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769848    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769851    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769854    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769856    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769859    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769862    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769865    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769868    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 11:33:06.779395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769870    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769873    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769877    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769880    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769883    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769885    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769888    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769891    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769893    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769896    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769900    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769904    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769907    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769910    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769914    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769917    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769920    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769923    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769926    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769928    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 11:33:06.780113 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769931    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769934    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769936    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769939    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769942    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769944    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769947    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769950    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769952    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769955    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769958    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769961    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769964    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769967    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769969    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769973    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769976    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769978    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769981    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 11:33:06.780740 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:06.769984    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.769989    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.770102    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.772197    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.773685    2558 server.go:1019] "Starting client certificate rotation"
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.773794    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.773873    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.776489    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 11:33:06.781275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.779351    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 11:33:06.788311 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.788292    2558 log.go:25] "Validated CRI v1 runtime API"
Aug 04 11:33:06.790529 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.790512    2558 log.go:25] "Validated CRI v1 image API"
Aug 04 11:33:06.791478 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.791461    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 11:33:06.797917 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.797892    2558 fs.go:135] Filesystem UUIDs: map[5581d540-4b74-4a28-afa5-ab11e8d8991f:/dev/nvme0n1p4 74a7c41d-d97a-438e-8e86-4317118f354d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Aug 04 11:33:06.797996 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.797915    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 11:33:06.803396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.803253    2558 manager.go:216] Machine: {Timestamp:2026-08-04 11:33:06.801827923 +0000 UTC m=+0.372740721 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103806 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26a50f6a0ac543665e775eb630e731 SystemUUID:ec26a50f-6a0a-c543-665e-775eb630e731 BootID:09e886ea-0ad1-42b7-85db-77ed6acc2859 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d0:73:93:e1:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d0:73:93:e1:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:26:1f:fc:cb:72 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 11:33:06.803396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.803383    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 11:33:06.803572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.803496    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 11:33:06.804341 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.804323    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 11:33:06.807326 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.807301    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 11:33:06.807493 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.807328    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 11:33:06.809739 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.809729    2558 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 11:33:06.809774 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.809741    2558 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 11:33:06.809774 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.809749    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 11:33:06.811506 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.811495    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 11:33:06.812973 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.812962    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 11:33:06.817484 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817472    2558 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 11:33:06.817608 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817597    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 11:33:06.817654 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817629    2558 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 11:33:06.817654 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817641    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 11:33:06.817724 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817655    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 11:33:06.817724 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817666    2558 kubelet.go:398] "Adding apiserver pod source"
Aug 04 11:33:06.817724 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.817674    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 11:33:06.818018 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.818007    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 11:33:06.818055 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.818026    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 11:33:06.821365 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.821334    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 11:33:06.822985 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.822969    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 11:33:06.823046 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.823003    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 11:33:06.824048 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824032    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 11:33:06.824113 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824058    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 11:33:06.824113 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824073    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 11:33:06.824113 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824085    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 11:33:06.824113 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824098    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 11:33:06.824113 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824105    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 11:33:06.824113 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824114    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 11:33:06.824285 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824120    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 11:33:06.824285 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824126    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 11:33:06.824285 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824133    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 11:33:06.824285 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824150    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 11:33:06.824285 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824156    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 11:33:06.824974 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824961    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 11:33:06.825013 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.824975    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 11:33:06.826968 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.826908    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 11:33:06.826968 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.826922    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 11:33:06.827105 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.827056    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 11:33:06.829178 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.829158    2558 server.go:1268] "Started kubelet"
Aug 04 11:33:06.829275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.829219    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 11:33:06.829415 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.829306    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 11:33:06.829482 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.829447    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 11:33:06.830150 ip-10-0-129-11 systemd[1]: Started Kubernetes Kubelet.
Aug 04 11:33:06.831141 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.831118    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 11:33:06.832164 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.832148    2558 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 11:33:06.835073 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.835051    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 11:33:06.838649 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.838632    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 11:33:06.838649 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.838641    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 11:33:06.838763 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.838724    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:06.839329 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839309    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 11:33:06.839329 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839309    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 11:33:06.839499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839339    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 11:33:06.839499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839448    2558 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 11:33:06.839499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839459    2558 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 11:33:06.839610 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839572    2558 factory.go:55] Registering systemd factory
Aug 04 11:33:06.839610 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839586    2558 factory.go:223] Registration of the systemd container factory successfully
Aug 04 11:33:06.839838 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839818    2558 factory.go:153] Registering CRI-O factory
Aug 04 11:33:06.839838 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839838    2558 factory.go:223] Registration of the crio container factory successfully
Aug 04 11:33:06.839992 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839893    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 11:33:06.839992 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839923    2558 factory.go:103] Registering Raw factory
Aug 04 11:33:06.839992 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.839940    2558 manager.go:1196] Started watching for new ooms in manager
Aug 04 11:33:06.840475 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.840450    2558 manager.go:349] Starting recovery of all containers
Aug 04 11:33:06.849446 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.849412    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 11:33:06.849597 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.849566    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 11:33:06.850810 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.849596    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa77ef3b0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.82903032 +0000 UTC m=+0.399943121,LastTimestamp:2026-08-04 11:33:06.82903032 +0000 UTC m=+0.399943121,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.851841 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.851823    2558 manager.go:354] Recovery completed
Aug 04 11:33:06.853114 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.853090    2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Aug 04 11:33:06.856240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.856220    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:06.858116 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858097    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:06.858166 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858132    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:06.858166 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858146    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:06.858756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858739    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 11:33:06.858756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858756    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 11:33:06.858887 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858747    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:06.858887 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.858772    2558 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 11:33:06.859523 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.859507    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:06.859607 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.859539    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:06.859607 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.859554    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:06.860087 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.860072    2558 policy_none.go:49] "None policy: Start"
Aug 04 11:33:06.860138 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.860089    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 11:33:06.860138 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.860100    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 11:33:06.861237 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.861226    2558 policy_none.go:47] "Start"
Aug 04 11:33:06.871645 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.871484    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa93acb10  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.858117904 +0000 UTC m=+0.429030704,LastTimestamp:2026-08-04 11:33:06.858117904 +0000 UTC m=+0.429030704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.877960 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.877900    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa93b1689  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.858137225 +0000 UTC m=+0.429050023,LastTimestamp:2026-08-04 11:33:06.858137225 +0000 UTC m=+0.429050023,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.881147 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.881080    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa93b4bb1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-11.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.858150833 +0000 UTC m=+0.429063630,LastTimestamp:2026-08-04 11:33:06.858150833 +0000 UTC m=+0.429063630,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.888127 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.888060    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-11.ec2.internal.18c896eaa93acb10\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa93acb10  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.858117904 +0000 UTC m=+0.429030704,LastTimestamp:2026-08-04 11:33:06.859521694 +0000 UTC m=+0.430434500,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.892844 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.892775    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-11.ec2.internal.18c896eaa93b1689\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa93b1689  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.858137225 +0000 UTC m=+0.429050023,LastTimestamp:2026-08-04 11:33:06.859545906 +0000 UTC m=+0.430458708,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.893060 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.893037    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7drp4"
Aug 04 11:33:06.897111 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.897042    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-11.ec2.internal.18c896eaa93b4bb1\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaa93b4bb1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-11.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.858150833 +0000 UTC m=+0.429063630,LastTimestamp:2026-08-04 11:33:06.859560403 +0000 UTC m=+0.430473202,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.899070 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.899053    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7drp4"
Aug 04 11:33:06.901039 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901023    2558 manager.go:337] "Starting Device Plugin manager"
Aug 04 11:33:06.901133 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.901064    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 11:33:06.901133 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901078    2558 server.go:87] "Starting device plugin registration server"
Aug 04 11:33:06.901405 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901392    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 11:33:06.901495 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901407    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 11:33:06.901545 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901498    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 11:33:06.901648 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901605    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 11:33:06.901648 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.901618    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 11:33:06.902268 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.902242    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 11:33:06.902382 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.902303    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:06.905276 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.905197    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-11.ec2.internal.18c896eaabf169ad  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-11.ec2.internal,UID:ip-10-0-129-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-11.ec2.internal,},FirstTimestamp:2026-08-04 11:33:06.903640493 +0000 UTC m=+0.474553281,LastTimestamp:2026-08-04 11:33:06.903640493 +0000 UTC m=+0.474553281,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-11.ec2.internal,}"
Aug 04 11:33:06.986056 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.986018    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 11:33:06.987340 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.987317    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 11:33:06.987446 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.987384    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 11:33:06.987446 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.987420    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 11:33:06.987546 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:06.987466    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 11:33:06.998192 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:06.998169    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 11:33:07.002463 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.002450    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:07.003225 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.003209    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:07.003301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.003242    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:07.003301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.003254    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:07.003301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.003279    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.014864 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.014845    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.014932 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.014866    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-11.ec2.internal\": node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.045366 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.045335    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.088248 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.088199    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"]
Aug 04 11:33:07.088374 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.088310    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:07.089371 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.089354    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:07.089460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.089386    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:07.089460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.089399    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:07.090682 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.090669    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:07.090835 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.090820    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.090896 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.090852    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:07.091418 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.091401    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:07.091510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.091431    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:07.091510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.091445    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:07.091614 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.091509    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:07.091614 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.091530    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:07.091614 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.091540    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:07.092906 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.092892    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.092962 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.092920    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 11:33:07.093662 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.093647    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 11:33:07.093734 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.093670    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 11:33:07.093734 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.093680    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeHasSufficientPID"
Aug 04 11:33:07.122125 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.122101    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-11.ec2.internal\" not found" node="ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.126980 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.126964    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-11.ec2.internal\" not found" node="ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.145518 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.145496    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.241180 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.241147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ea7ce17e14565481341beac22d82055-config\") pod \"kube-apiserver-proxy-ip-10-0-129-11.ec2.internal\" (UID: \"1ea7ce17e14565481341beac22d82055\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.241315 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.241182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/255b91d76077e362f2d44ad63f17ff16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal\" (UID: \"255b91d76077e362f2d44ad63f17ff16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.241315 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.241202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/255b91d76077e362f2d44ad63f17ff16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal\" (UID: \"255b91d76077e362f2d44ad63f17ff16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.246659 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.246640    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.341470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.341409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ea7ce17e14565481341beac22d82055-config\") pod \"kube-apiserver-proxy-ip-10-0-129-11.ec2.internal\" (UID: \"1ea7ce17e14565481341beac22d82055\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.341470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.341443    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/255b91d76077e362f2d44ad63f17ff16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal\" (UID: \"255b91d76077e362f2d44ad63f17ff16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.341588 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.341497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/255b91d76077e362f2d44ad63f17ff16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal\" (UID: \"255b91d76077e362f2d44ad63f17ff16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.341588 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.341506    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ea7ce17e14565481341beac22d82055-config\") pod \"kube-apiserver-proxy-ip-10-0-129-11.ec2.internal\" (UID: \"1ea7ce17e14565481341beac22d82055\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.341588 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.341540    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/255b91d76077e362f2d44ad63f17ff16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal\" (UID: \"255b91d76077e362f2d44ad63f17ff16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.341588 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.341568    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/255b91d76077e362f2d44ad63f17ff16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal\" (UID: \"255b91d76077e362f2d44ad63f17ff16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.346838 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.346818    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.428699 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.428674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.429831 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.429816    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.447325 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.447303    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.548045 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.548004    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.648837 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.648775    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.748910 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:07.748864    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-11.ec2.internal\" not found"
Aug 04 11:33:07.773050 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.773025    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 11:33:07.773200 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.773177    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 11:33:07.820546 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.820517    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 11:33:07.839048 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.839025    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.839048 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.839041    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 11:33:07.850939 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.850900    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 11:33:07.855366 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.855348    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal"
Aug 04 11:33:07.862282 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.862263    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 11:33:07.879027 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.879007    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 11:33:07.904251 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.904188    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 11:28:07 +0000 UTC" deadline="2028-04-26 01:40:20.57015341 +0000 UTC"
Aug 04 11:33:07.904251 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.904211    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15134h7m12.665945264s"
Aug 04 11:33:07.904586 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.904568    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dltrs"
Aug 04 11:33:07.916596 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:07.916580    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dltrs"
Aug 04 11:33:08.041606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.041575    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 11:33:08.089678 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:08.089641    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255b91d76077e362f2d44ad63f17ff16.slice/crio-73ae7c4487d73254e70dc5cd28a6fcd44f5b51acbe6c4ffcdf634db3af8f82d8 WatchSource:0}: Error finding container 73ae7c4487d73254e70dc5cd28a6fcd44f5b51acbe6c4ffcdf634db3af8f82d8: Status 404 returned error can't find the container with id 73ae7c4487d73254e70dc5cd28a6fcd44f5b51acbe6c4ffcdf634db3af8f82d8
Aug 04 11:33:08.095538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.095521    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 11:33:08.127624 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:08.127587    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea7ce17e14565481341beac22d82055.slice/crio-09d864312dc7c675b935510c05fbe89302dd9cc4e31779bc2d3c5f71833ded9e WatchSource:0}: Error finding container 09d864312dc7c675b935510c05fbe89302dd9cc4e31779bc2d3c5f71833ded9e: Status 404 returned error can't find the container with id 09d864312dc7c675b935510c05fbe89302dd9cc4e31779bc2d3c5f71833ded9e
Aug 04 11:33:08.294025 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.294004    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 11:33:08.819841 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.819810    2558 apiserver.go:52] "Watching apiserver"
Aug 04 11:33:08.829760 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.829614    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 11:33:08.830157 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.830130    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-467j9","kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal","openshift-image-registry/node-ca-tbxgw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-cbd22","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7","openshift-cluster-node-tuning-operator/tuned-fqkgr","openshift-dns/node-resolver-8klxl","openshift-multus/multus-additional-cni-plugins-c8pjg","openshift-multus/multus-rk8d8","openshift-multus/network-metrics-daemon-s7lbd","openshift-network-diagnostics/network-check-target-gb5hk","openshift-network-operator/iptables-alerter-m5lv9","kube-system/global-pull-secret-syncer-j6nsk"]
Aug 04 11:33:08.832162 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.832142    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.833601 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.833581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.835386 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.834866    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 11:33:08.835386 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.834953    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.835386 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.835171    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 11:33:08.835386 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.835379    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8r8kt\""
Aug 04 11:33:08.835709 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.835539    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.835709 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.835644    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.836964 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.836928    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.837061 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.836990    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.837255 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.837238    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rzxnk\""
Aug 04 11:33:08.837530 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.837502    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 11:33:08.840346 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.840322    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 11:33:08.840633 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.840617    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.840881 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.840865    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.841050 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.841024    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sq4gc\""
Aug 04 11:33:08.841106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.840868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.841220 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:08.841195    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:08.841775 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.841755    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.844677 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.844658    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 11:33:08.844788 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.844684    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.845678 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.845660    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 11:33:08.846391 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.845946    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5bck5\""
Aug 04 11:33:08.846391 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.846270    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.846771 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.846749    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.847940 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.847579    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 11:33:08.847940 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.847919    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.847940 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.847934    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 11:33:08.849517 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.849497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.849993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.849973    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sb42h\""
Aug 04 11:33:08.850127 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.850110    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.850341 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.850315    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 11:33:08.850489 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.850317    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.850863 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.850845    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:08.851407 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-netns\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.851735 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851713    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-etc-kubernetes\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.851837 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851747    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-systemd\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.851837 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqprh\" (UniqueName: \"kubernetes.io/projected/2f3f1bb4-c75e-4b61-9db9-034c63028124-kube-api-access-hqprh\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.851837 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851669    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nf4q8\""
Aug 04 11:33:08.851837 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-kubelet-config\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.851837 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-dbus\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-ovnkube-config\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-cni-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851670    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-os-release\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851945    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-etc-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-ovn\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.851997    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-run-ovn-kubernetes\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852021    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-kubelet\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-hostroot\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-iptables-alerter-script\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p48bh\" (UniqueName: \"kubernetes.io/projected/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-kube-api-access-p48bh\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.852106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852107    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-slash\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852184    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f3f1bb4-c75e-4b61-9db9-034c63028124-host\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f3f1bb4-c75e-4b61-9db9-034c63028124-serviceca\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852219    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852239    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-var-lib-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852266    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-log-socket\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852292    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b036cb3-960c-44b5-a80c-49e266f49ab9-cni-binary-copy\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852319    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-k8s-cni-cncf-io\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852376    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-cni-netd\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852437    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-ovnkube-script-lib\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852517    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-conf-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852583    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-daemon-config\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbs94\" (UniqueName: \"kubernetes.io/projected/0b036cb3-960c-44b5-a80c-49e266f49ab9-kube-api-access-rbs94\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852697    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-kubelet\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.852736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-node-log\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852771    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7d80118-a3de-48a5-b640-5356cc910710-ovn-node-metrics-cert\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852844    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fkpt\" (UniqueName: \"kubernetes.io/projected/d7d80118-a3de-48a5-b640-5356cc910710-kube-api-access-5fkpt\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852897    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-socket-dir-parent\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852916    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hp5mm\""
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852924    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-host-slash\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852952    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.852981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-run-netns\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853035    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-env-overrides\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853062    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-cnibin\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853090    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-cni-multus\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853150    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-multus-certs\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-systemd-units\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853220    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853243    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-cni-bin\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-system-cni-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.853584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853299    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-cni-bin\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.854560 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853790    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 11:33:08.854560 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853968    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hpwf7\""
Aug 04 11:33:08.854560 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.853973    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 11:33:08.854690 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.854616    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:08.854727 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:08.854680    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:08.856260 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.856239    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:08.856375 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:08.856311    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:08.859079 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.858865    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 11:33:08.859079 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.858913    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 11:33:08.859357 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.859333    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nsnmf\""
Aug 04 11:33:08.918478 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.918359    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 11:28:08 +0000 UTC" deadline="2028-01-06 07:24:58.998766386 +0000 UTC"
Aug 04 11:33:08.918478 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.918391    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12475h51m50.080380118s"
Aug 04 11:33:08.941685 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.941595    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 11:33:08.953660 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-kubelet\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.953782 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fkpt\" (UniqueName: \"kubernetes.io/projected/d7d80118-a3de-48a5-b640-5356cc910710-kube-api-access-5fkpt\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.953782 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953699    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-socket-dir-parent\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.953782 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953730    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgttd\" (UniqueName: \"kubernetes.io/projected/9f82d5f7-dfb7-4745-8b34-544761a0f69d-kube-api-access-fgttd\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.953782 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-run\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2sck\" (UniqueName: \"kubernetes.io/projected/91581844-de78-4503-98e6-fd92d64fa837-kube-api-access-r2sck\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-systemd-units\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-cni-bin\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-cni-bin\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953966    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cnibin\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.953991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysconfig\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-sys\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.954076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-lib-modules\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954075    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-host\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954110    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cni-binary-copy\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954141    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954175    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-var-lib-kubelet\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954204    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c69b084c-b802-42fb-bfa4-865368a83757-agent-certs\") pod \"konnectivity-agent-467j9\" (UID: \"c69b084c-b802-42fb-bfa4-865368a83757\") " pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954246    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkz2\" (UniqueName: \"kubernetes.io/projected/34457f78-d6ea-4f3b-9061-19bc08898912-kube-api-access-7qkz2\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954377    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-cni-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-os-release\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-cni-bin\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-kubelet\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954492    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-registration-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-etc-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-ovn\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954601    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-run-ovn-kubernetes\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.954683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-os-release\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954673    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysctl-conf\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34457f78-d6ea-4f3b-9061-19bc08898912-tmp-dir\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954745    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f3f1bb4-c75e-4b61-9db9-034c63028124-host\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954770    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-k8s-cni-cncf-io\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954813    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-etc-selinux\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954827    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-socket-dir-parent\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-cni-netd\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.954954    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-cni-bin\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:08.954999    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:08.955101    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:33:09.455076582 +0000 UTC m=+3.025989381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955240    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-systemd-units\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955272    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-ovnkube-script-lib\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955313    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-os-release\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955332    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-daemon-config\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.955553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955387    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-kubelet\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955388    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955429    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-cni-netd\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955500    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-etc-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-kubelet\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-ovn\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-run-ovn-kubernetes\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955720    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f3f1bb4-c75e-4b61-9db9-034c63028124-host\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955762    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-k8s-cni-cncf-io\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-cni-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.955429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-device-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956100    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-sys-fs\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-node-log\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7d80118-a3de-48a5-b640-5356cc910710-ovn-node-metrics-cert\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956217    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-host-slash\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956242    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-kubernetes\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.956470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956280    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c69b084c-b802-42fb-bfa4-865368a83757-konnectivity-ca\") pod \"konnectivity-agent-467j9\" (UID: \"c69b084c-b802-42fb-bfa4-865368a83757\") " pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956336    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-ovnkube-script-lib\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-run-netns\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956399    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-env-overrides\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-cnibin\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-cni-multus\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-multus-certs\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-tuned\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f64e0f15-2462-46b2-af7c-5ad9061f593f-tmp\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-system-cni-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956589    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-node-log\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-systemd\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-netns\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-etc-kubernetes\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-systemd\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.957427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqprh\" (UniqueName: \"kubernetes.io/projected/2f3f1bb4-c75e-4b61-9db9-034c63028124-kube-api-access-hqprh\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-kubelet-config\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-dbus\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-ovnkube-config\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-hostroot\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-iptables-alerter-script\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956987    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.956997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p48bh\" (UniqueName: \"kubernetes.io/projected/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-kube-api-access-p48bh\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957059    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vgxt\" (UniqueName: \"kubernetes.io/projected/f64e0f15-2462-46b2-af7c-5ad9061f593f-kube-api-access-5vgxt\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-socket-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957108    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34457f78-d6ea-4f3b-9061-19bc08898912-hosts-file\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-slash\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f3f1bb4-c75e-4b61-9db9-034c63028124-serviceca\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957273    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-var-lib-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-log-socket\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b036cb3-960c-44b5-a80c-49e266f49ab9-cni-binary-copy\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.958265 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957393    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-host-slash\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-conf-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957466    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbs94\" (UniqueName: \"kubernetes.io/projected/0b036cb3-960c-44b5-a80c-49e266f49ab9-kube-api-access-rbs94\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957496    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-system-cni-dir\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-modprobe-d\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysctl-d\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlpgs\" (UniqueName: \"kubernetes.io/projected/cd9918b3-753f-4772-a082-1e922de1494d-kube-api-access-mlpgs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957698    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-conf-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.957710    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-run-netns\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958082    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b036cb3-960c-44b5-a80c-49e266f49ab9-multus-daemon-config\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-env-overrides\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958374    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-kubelet-config\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958505    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-cnibin\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-dbus\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958623    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-run-systemd\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-netns\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-etc-kubernetes\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958700    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-hostroot\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.960170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958766    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-run-multus-certs\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.958761    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-host-var-lib-cni-multus\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-var-lib-openvswitch\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959198    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-host-slash\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959246    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7d80118-a3de-48a5-b640-5356cc910710-log-socket\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959307    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7d80118-a3de-48a5-b640-5356cc910710-ovnkube-config\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-iptables-alerter-script\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959463    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f3f1bb4-c75e-4b61-9db9-034c63028124-serviceca\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959629    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b036cb3-960c-44b5-a80c-49e266f49ab9-system-cni-dir\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.959797    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b036cb3-960c-44b5-a80c-49e266f49ab9-cni-binary-copy\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.961613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.961243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7d80118-a3de-48a5-b640-5356cc910710-ovn-node-metrics-cert\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.965647 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.965610    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fkpt\" (UniqueName: \"kubernetes.io/projected/d7d80118-a3de-48a5-b640-5356cc910710-kube-api-access-5fkpt\") pod \"ovnkube-node-cbd22\" (UID: \"d7d80118-a3de-48a5-b640-5356cc910710\") " pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:08.973589 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.973487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p48bh\" (UniqueName: \"kubernetes.io/projected/dda3ac9d-4ad9-4bef-92fb-190b001daa4e-kube-api-access-p48bh\") pod \"iptables-alerter-m5lv9\" (UID: \"dda3ac9d-4ad9-4bef-92fb-190b001daa4e\") " pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:08.973819 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.973798    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqprh\" (UniqueName: \"kubernetes.io/projected/2f3f1bb4-c75e-4b61-9db9-034c63028124-kube-api-access-hqprh\") pod \"node-ca-tbxgw\" (UID: \"2f3f1bb4-c75e-4b61-9db9-034c63028124\") " pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:08.979448 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.978811    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbs94\" (UniqueName: \"kubernetes.io/projected/0b036cb3-960c-44b5-a80c-49e266f49ab9-kube-api-access-rbs94\") pod \"multus-rk8d8\" (UID: \"0b036cb3-960c-44b5-a80c-49e266f49ab9\") " pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:08.994961 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.994875    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal" event={"ID":"1ea7ce17e14565481341beac22d82055","Type":"ContainerStarted","Data":"09d864312dc7c675b935510c05fbe89302dd9cc4e31779bc2d3c5f71833ded9e"}
Aug 04 11:33:08.996629 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:08.996598    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal" event={"ID":"255b91d76077e362f2d44ad63f17ff16","Type":"ContainerStarted","Data":"73ae7c4487d73254e70dc5cd28a6fcd44f5b51acbe6c4ffcdf634db3af8f82d8"}
Aug 04 11:33:09.058566 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34457f78-d6ea-4f3b-9061-19bc08898912-hosts-file\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.058566 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058555    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-system-cni-dir\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058586    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-modprobe-d\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysctl-d\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlpgs\" (UniqueName: \"kubernetes.io/projected/cd9918b3-753f-4772-a082-1e922de1494d-kube-api-access-mlpgs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgttd\" (UniqueName: \"kubernetes.io/projected/9f82d5f7-dfb7-4745-8b34-544761a0f69d-kube-api-access-fgttd\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058698    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-run\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2sck\" (UniqueName: \"kubernetes.io/projected/91581844-de78-4503-98e6-fd92d64fa837-kube-api-access-r2sck\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058747    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34457f78-d6ea-4f3b-9061-19bc08898912-hosts-file\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.058821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058772    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cnibin\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058827    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-system-cni-dir\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058834    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cnibin\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysconfig\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-sys\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-lib-modules\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-host\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cni-binary-copy\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-var-lib-kubelet\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c69b084c-b802-42fb-bfa4-865368a83757-agent-certs\") pod \"konnectivity-agent-467j9\" (UID: \"c69b084c-b802-42fb-bfa4-865368a83757\") " pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkz2\" (UniqueName: \"kubernetes.io/projected/34457f78-d6ea-4f3b-9061-19bc08898912-kube-api-access-7qkz2\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-registration-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-os-release\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysctl-conf\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.059234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34457f78-d6ea-4f3b-9061-19bc08898912-tmp-dir\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-etc-selinux\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059251    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-run\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059309    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-device-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-sys-fs\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-kubernetes\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059406    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c69b084c-b802-42fb-bfa4-865368a83757-konnectivity-ca\") pod \"konnectivity-agent-467j9\" (UID: \"c69b084c-b802-42fb-bfa4-865368a83757\") " pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-modprobe-d\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059436    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-tuned\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f64e0f15-2462-46b2-af7c-5ad9061f593f-tmp\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059526    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysconfig\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059550    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-systemd\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059592    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-sys\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.060037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059657    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-systemd\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059672    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vgxt\" (UniqueName: \"kubernetes.io/projected/f64e0f15-2462-46b2-af7c-5ad9061f593f-kube-api-access-5vgxt\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-socket-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059791    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-host\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059853    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-socket-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.059943    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-sys-fs\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.059949    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060008    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-kubernetes\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.060020    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:33:09.559994153 +0000 UTC m=+3.130906952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-etc-selinux\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060463    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34457f78-d6ea-4f3b-9061-19bc08898912-tmp-dir\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-device-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-lib-modules\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.058996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysctl-d\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060757    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.060911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060761    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c69b084c-b802-42fb-bfa4-865368a83757-konnectivity-ca\") pod \"konnectivity-agent-467j9\" (UID: \"c69b084c-b802-42fb-bfa4-865368a83757\") " pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:09.061866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-var-lib-kubelet\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.061866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060848    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91581844-de78-4503-98e6-fd92d64fa837-registration-dir\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.061866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-os-release\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.061866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060891    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-sysctl-conf\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.061866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.060975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f82d5f7-dfb7-4745-8b34-544761a0f69d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.061866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.061240    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f82d5f7-dfb7-4745-8b34-544761a0f69d-cni-binary-copy\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.063328 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.063302    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c69b084c-b802-42fb-bfa4-865368a83757-agent-certs\") pod \"konnectivity-agent-467j9\" (UID: \"c69b084c-b802-42fb-bfa4-865368a83757\") " pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:09.063451 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.063396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f64e0f15-2462-46b2-af7c-5ad9061f593f-tmp\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.064662 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.064640    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f64e0f15-2462-46b2-af7c-5ad9061f593f-etc-tuned\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.070843 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.070785    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vgxt\" (UniqueName: \"kubernetes.io/projected/f64e0f15-2462-46b2-af7c-5ad9061f593f-kube-api-access-5vgxt\") pod \"tuned-fqkgr\" (UID: \"f64e0f15-2462-46b2-af7c-5ad9061f593f\") " pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.073156 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.072412    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:09.073156 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.072435    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:09.073156 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.072450    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:09.073156 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.072590    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:09.572568016 +0000 UTC m=+3.143480821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:09.075416 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.075382    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgttd\" (UniqueName: \"kubernetes.io/projected/9f82d5f7-dfb7-4745-8b34-544761a0f69d-kube-api-access-fgttd\") pod \"multus-additional-cni-plugins-c8pjg\" (UID: \"9f82d5f7-dfb7-4745-8b34-544761a0f69d\") " pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.077861 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.077832    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2sck\" (UniqueName: \"kubernetes.io/projected/91581844-de78-4503-98e6-fd92d64fa837-kube-api-access-r2sck\") pod \"aws-ebs-csi-driver-node-gd5k7\" (UID: \"91581844-de78-4503-98e6-fd92d64fa837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.078496 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.078468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkz2\" (UniqueName: \"kubernetes.io/projected/34457f78-d6ea-4f3b-9061-19bc08898912-kube-api-access-7qkz2\") pod \"node-resolver-8klxl\" (UID: \"34457f78-d6ea-4f3b-9061-19bc08898912\") " pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.079518 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.079468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlpgs\" (UniqueName: \"kubernetes.io/projected/cd9918b3-753f-4772-a082-1e922de1494d-kube-api-access-mlpgs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:09.150384 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.150350    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rk8d8"
Aug 04 11:33:09.158523 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.158307    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5lv9"
Aug 04 11:33:09.160912 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.160866    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b036cb3_960c_44b5_a80c_49e266f49ab9.slice/crio-02e7f432462476a25351b03e9653ed13ce980cef1765c240b5304e3422893a5b WatchSource:0}: Error finding container 02e7f432462476a25351b03e9653ed13ce980cef1765c240b5304e3422893a5b: Status 404 returned error can't find the container with id 02e7f432462476a25351b03e9653ed13ce980cef1765c240b5304e3422893a5b
Aug 04 11:33:09.167988 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.167961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tbxgw"
Aug 04 11:33:09.168450 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.168425    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda3ac9d_4ad9_4bef_92fb_190b001daa4e.slice/crio-2177262da56fb2716532b5cb24301bdd7b36494328bd952f1db7dd8e7eba4b70 WatchSource:0}: Error finding container 2177262da56fb2716532b5cb24301bdd7b36494328bd952f1db7dd8e7eba4b70: Status 404 returned error can't find the container with id 2177262da56fb2716532b5cb24301bdd7b36494328bd952f1db7dd8e7eba4b70
Aug 04 11:33:09.178158 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.177975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:09.182581 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.182554    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3f1bb4_c75e_4b61_9db9_034c63028124.slice/crio-e8d326e7f71f1b82e19891d0a864b8e3a0dd9b484081bed27b5b01f20c73c2dc WatchSource:0}: Error finding container e8d326e7f71f1b82e19891d0a864b8e3a0dd9b484081bed27b5b01f20c73c2dc: Status 404 returned error can't find the container with id e8d326e7f71f1b82e19891d0a864b8e3a0dd9b484081bed27b5b01f20c73c2dc
Aug 04 11:33:09.187854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.187817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7"
Aug 04 11:33:09.189031 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.189000    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d80118_a3de_48a5_b640_5356cc910710.slice/crio-e1a42ebe3ae388b602452a4ce88c641c932795f7a4c16aaa72e8ee19d8338666 WatchSource:0}: Error finding container e1a42ebe3ae388b602452a4ce88c641c932795f7a4c16aaa72e8ee19d8338666: Status 404 returned error can't find the container with id e1a42ebe3ae388b602452a4ce88c641c932795f7a4c16aaa72e8ee19d8338666
Aug 04 11:33:09.195888 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.195722    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c8pjg"
Aug 04 11:33:09.196898 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.196874    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91581844_de78_4503_98e6_fd92d64fa837.slice/crio-72f964453b5d3b9c9a3774db8bf70e6bc070a879e9de9ed7c7b69177dce21222 WatchSource:0}: Error finding container 72f964453b5d3b9c9a3774db8bf70e6bc070a879e9de9ed7c7b69177dce21222: Status 404 returned error can't find the container with id 72f964453b5d3b9c9a3774db8bf70e6bc070a879e9de9ed7c7b69177dce21222
Aug 04 11:33:09.204756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.204734    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fqkgr"
Aug 04 11:33:09.205244 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.205218    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f82d5f7_dfb7_4745_8b34_544761a0f69d.slice/crio-6e0d4c0f6bdf1729ef4b1b213583643f48628c5ba68a2830f16d02c0d2ff906d WatchSource:0}: Error finding container 6e0d4c0f6bdf1729ef4b1b213583643f48628c5ba68a2830f16d02c0d2ff906d: Status 404 returned error can't find the container with id 6e0d4c0f6bdf1729ef4b1b213583643f48628c5ba68a2830f16d02c0d2ff906d
Aug 04 11:33:09.213132 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.213016    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8klxl"
Aug 04 11:33:09.215071 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.215033    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64e0f15_2462_46b2_af7c_5ad9061f593f.slice/crio-2a43aeadc75456350278cc0fdb7d137715ad8b259b70d94e84736af3a43d8acd WatchSource:0}: Error finding container 2a43aeadc75456350278cc0fdb7d137715ad8b259b70d94e84736af3a43d8acd: Status 404 returned error can't find the container with id 2a43aeadc75456350278cc0fdb7d137715ad8b259b70d94e84736af3a43d8acd
Aug 04 11:33:09.221583 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.221477    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:09.225248 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.225216    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34457f78_d6ea_4f3b_9061_19bc08898912.slice/crio-04283ff38bbcf9c7e7b15596265dd90520501780467156102729ff24a37afbea WatchSource:0}: Error finding container 04283ff38bbcf9c7e7b15596265dd90520501780467156102729ff24a37afbea: Status 404 returned error can't find the container with id 04283ff38bbcf9c7e7b15596265dd90520501780467156102729ff24a37afbea
Aug 04 11:33:09.233720 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:09.233689    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69b084c_b802_42fb_bfa4_865368a83757.slice/crio-b32eedd9f05bbad28c3ade6169e2c81b234f7606206a668896f106e30d3b1d17 WatchSource:0}: Error finding container b32eedd9f05bbad28c3ade6169e2c81b234f7606206a668896f106e30d3b1d17: Status 404 returned error can't find the container with id b32eedd9f05bbad28c3ade6169e2c81b234f7606206a668896f106e30d3b1d17
Aug 04 11:33:09.358946 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.358863    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 11:33:09.462636 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.462600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:09.462838 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.462789    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:09.462907 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.462877    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:33:10.462852794 +0000 UTC m=+4.033765586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:09.564062 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.564027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:09.564239 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.564191    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:09.564304 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.564253    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:33:10.564238926 +0000 UTC m=+4.135151713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:09.664872 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.664788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:09.665038 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.664940    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:09.665038 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.664958    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:09.665038 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.664970    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:09.665038 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.665025    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:10.66500909 +0000 UTC m=+4.235921873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:09.918679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.918567    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 11:28:08 +0000 UTC" deadline="2028-05-02 04:03:02.851305394 +0000 UTC"
Aug 04 11:33:09.918679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.918622    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15280h29m52.93268795s"
Aug 04 11:33:09.988584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.988553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:09.988752 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:09.988666    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:09.999340 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:09.999305    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-467j9" event={"ID":"c69b084c-b802-42fb-bfa4-865368a83757","Type":"ContainerStarted","Data":"b32eedd9f05bbad28c3ade6169e2c81b234f7606206a668896f106e30d3b1d17"}
Aug 04 11:33:10.000537 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.000505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8klxl" event={"ID":"34457f78-d6ea-4f3b-9061-19bc08898912","Type":"ContainerStarted","Data":"04283ff38bbcf9c7e7b15596265dd90520501780467156102729ff24a37afbea"}
Aug 04 11:33:10.001970 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.001945    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fqkgr" event={"ID":"f64e0f15-2462-46b2-af7c-5ad9061f593f","Type":"ContainerStarted","Data":"2a43aeadc75456350278cc0fdb7d137715ad8b259b70d94e84736af3a43d8acd"}
Aug 04 11:33:10.007146 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.006970    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7" event={"ID":"91581844-de78-4503-98e6-fd92d64fa837","Type":"ContainerStarted","Data":"72f964453b5d3b9c9a3774db8bf70e6bc070a879e9de9ed7c7b69177dce21222"}
Aug 04 11:33:10.008949 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.008924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"e1a42ebe3ae388b602452a4ce88c641c932795f7a4c16aaa72e8ee19d8338666"}
Aug 04 11:33:10.010284 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.010260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rk8d8" event={"ID":"0b036cb3-960c-44b5-a80c-49e266f49ab9","Type":"ContainerStarted","Data":"02e7f432462476a25351b03e9653ed13ce980cef1765c240b5304e3422893a5b"}
Aug 04 11:33:10.011589 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.011564    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerStarted","Data":"6e0d4c0f6bdf1729ef4b1b213583643f48628c5ba68a2830f16d02c0d2ff906d"}
Aug 04 11:33:10.012920 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.012894    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tbxgw" event={"ID":"2f3f1bb4-c75e-4b61-9db9-034c63028124","Type":"ContainerStarted","Data":"e8d326e7f71f1b82e19891d0a864b8e3a0dd9b484081bed27b5b01f20c73c2dc"}
Aug 04 11:33:10.014102 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.014083    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5lv9" event={"ID":"dda3ac9d-4ad9-4bef-92fb-190b001daa4e","Type":"ContainerStarted","Data":"2177262da56fb2716532b5cb24301bdd7b36494328bd952f1db7dd8e7eba4b70"}
Aug 04 11:33:10.471759 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.471041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:10.471759 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.471276    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:10.471759 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.471368    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:33:12.471346715 +0000 UTC m=+6.042259517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:10.571905 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.571862    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:10.572118 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.572035    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:10.572118 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.572105    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:33:12.572083365 +0000 UTC m=+6.142996155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:10.673334 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.673282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:10.673530 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.673496    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:10.673530 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.673517    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:10.673530 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.673531    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:10.673790 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.673597    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:12.673577007 +0000 UTC m=+6.244489793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:10.989222 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.988512    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:10.989222 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.988833    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:10.989815 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:10.989222    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:10.989815 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:10.989375    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:11.987788 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:11.987697    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:11.988000 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:11.987840    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:12.048120 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.047126    2558 generic.go:356] "Generic (PLEG): container finished" podID="255b91d76077e362f2d44ad63f17ff16" containerID="71ed58cc065cd13563a5a47388ef575b7b8477945c46d76646e125a528d55539" exitCode=0
Aug 04 11:33:12.048120 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.047228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal" event={"ID":"255b91d76077e362f2d44ad63f17ff16","Type":"ContainerDied","Data":"71ed58cc065cd13563a5a47388ef575b7b8477945c46d76646e125a528d55539"}
Aug 04 11:33:12.058434 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.057935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal" event={"ID":"1ea7ce17e14565481341beac22d82055","Type":"ContainerStarted","Data":"9a5818f604cdd431b0a4408e00a4466644132de876f56f59baadbf5079159485"}
Aug 04 11:33:12.084165 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.082968    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-11.ec2.internal" podStartSLOduration=4.082948023 podStartE2EDuration="4.082948023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:12.082773348 +0000 UTC m=+5.653686157" watchObservedRunningTime="2026-08-04 11:33:12.082948023 +0000 UTC m=+5.653860828"
Aug 04 11:33:12.490660 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.490003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:12.490660 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.490197    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:12.490660 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.490269    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:33:16.490249389 +0000 UTC m=+10.061162195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:12.591674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.591583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:12.591850 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.591776    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:12.591922 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.591859    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:33:16.591836304 +0000 UTC m=+10.162749106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:12.692657 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.692616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:12.692866 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.692845    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:12.692934 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.692869    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:12.692934 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.692883    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:12.693042 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.692949    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:16.692927075 +0000 UTC m=+10.263839865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:12.988437 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.988339    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:12.988633 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:12.988431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:12.988633 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.988534    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:12.988633 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:12.988542    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:13.064858 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:13.064436    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal" event={"ID":"255b91d76077e362f2d44ad63f17ff16","Type":"ContainerStarted","Data":"74d71b8279680c3be09ab361968c0c0c8cf4ec67983f55028ab9a003eb374049"}
Aug 04 11:33:13.093651 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:13.093589    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-11.ec2.internal" podStartSLOduration=5.093562587 podStartE2EDuration="5.093562587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:13.092415896 +0000 UTC m=+6.663328704" watchObservedRunningTime="2026-08-04 11:33:13.093562587 +0000 UTC m=+6.664475400"
Aug 04 11:33:13.988551 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:13.988038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:13.988551 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:13.988173    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:14.988945 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:14.988402    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:14.988945 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:14.988518    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:14.988945 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:14.988847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:14.993349 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:14.990617    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:15.988803 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:15.988389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:15.988803 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:15.988504    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:16.527593 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:16.527391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:16.527593 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.527574    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:16.527836 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.527639    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:33:24.527619958 +0000 UTC m=+18.098532759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:16.628014 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:16.627975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:16.628191 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.628135    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:16.628266 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.628221    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:33:24.628198909 +0000 UTC m=+18.199111702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:16.728556 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:16.728523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:16.728713 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.728666    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:16.728713 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.728688    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:16.728713 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.728699    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:16.728848 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.728753    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:24.728738106 +0000 UTC m=+18.299650889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:16.990401 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:16.989703    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:16.990401 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.989825    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:16.990401 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:16.990203    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:16.990401 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:16.990306    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:17.988165 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:17.988133    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:17.988390 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:17.988271    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:18.991993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:18.991591    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:18.991993 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:18.991726    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:18.991993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:18.991762    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:18.991993 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:18.991909    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:19.988589 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:19.988557    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:19.988767 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:19.988678    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:20.988429 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:20.988341    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:20.988872 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:20.988345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:20.988872 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:20.988465    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:20.988872 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:20.988549    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:21.987949 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:21.987918    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:21.988111 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:21.988021    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:22.988166 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:22.988128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:22.988621 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:22.988134    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:22.988621 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:22.988266    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:22.988621 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:22.988322    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:23.987759 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:23.987720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:23.987979 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:23.987863    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:24.591677 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:24.591639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:24.592142 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.591822    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:24.592142 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.591904    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:33:40.591883359 +0000 UTC m=+34.162796144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:24.693128 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:24.693044    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:24.693290 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.693188    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:24.693290 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.693253    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:33:40.693234804 +0000 UTC m=+34.264147594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:24.793791 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:24.793751    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:24.793968 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.793915    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:24.793968 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.793936    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:24.793968 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.793947    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:24.794097 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.794008    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:40.793992068 +0000 UTC m=+34.364904871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:24.987807 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:24.987710    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:24.987961 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.987835    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:24.987961 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:24.987890    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:24.988088 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:24.988011    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:25.987774 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:25.987742    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:25.988168 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:25.987847    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:26.988858 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:26.988829    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:26.989326 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:26.988939    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:26.989326 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:26.988999    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:26.989326 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:26.989104    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:27.987801 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:27.987744    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:27.987989 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:27.987867    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:28.988423 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:28.988119    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:28.989025 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:28.988190    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:28.989025 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:28.988511    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:28.989025 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:28.988601    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:29.096884 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.096847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-467j9" event={"ID":"c69b084c-b802-42fb-bfa4-865368a83757","Type":"ContainerStarted","Data":"b067d2cb599c1795b47310d10a17310902346e0ca97f24c39503f6a2b8c224f0"}
Aug 04 11:33:29.098454 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.098428    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8klxl" event={"ID":"34457f78-d6ea-4f3b-9061-19bc08898912","Type":"ContainerStarted","Data":"cd8a314c228a826484a4106bfe8a91478dae20eaf8ebb839981d79f0770da164"}
Aug 04 11:33:29.100018 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.099993    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fqkgr" event={"ID":"f64e0f15-2462-46b2-af7c-5ad9061f593f","Type":"ContainerStarted","Data":"5380a8d138c3d808ac69431aac5d320f9bac4c98647cafdb9697ceaeaf3f35b4"}
Aug 04 11:33:29.101488 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.101465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7" event={"ID":"91581844-de78-4503-98e6-fd92d64fa837","Type":"ContainerStarted","Data":"b6a0c2cce6b8ef34291d9651a3b2d92d716588b5b550762df192f9c2384c0171"}
Aug 04 11:33:29.104599 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.104575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"1703b8d70200654942625353b53e76c6d489a31f5539fe6688312609f2c36d84"}
Aug 04 11:33:29.104694 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.104647    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"ba476d0d11b13aec679566df36e05ff6474cc8ddd1b2b811283f097546699ab9"}
Aug 04 11:33:29.104694 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.104687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"aa723b4490569581eadedb445cc04851c7b43b66cb6c0612117740d96b61a6c1"}
Aug 04 11:33:29.104804 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.104728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"0fdc7da6be7aecc8448ea15f29417c10f6b5db9e9d09b4a78ece3485ce5d00d6"}
Aug 04 11:33:29.104804 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.104758    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"7d847add8e6c853e7faefab412df99afd410c045a67c91e5761e13dbe3b597ce"}
Aug 04 11:33:29.104804 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.104789    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"ee45a8c924f6f2187162a57fac346f355312dba59872bccb9cbad0e5527bcf20"}
Aug 04 11:33:29.106005 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.105984    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rk8d8" event={"ID":"0b036cb3-960c-44b5-a80c-49e266f49ab9","Type":"ContainerStarted","Data":"fda31249ebff52de96c9c405c60635c73174d694dd7cd53ce2569a37e0a42e2b"}
Aug 04 11:33:29.107445 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.107423    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f82d5f7-dfb7-4745-8b34-544761a0f69d" containerID="9741cecde9a510cadf6f6b54b0869332f7152cfb588418751e67104d39fba5b6" exitCode=0
Aug 04 11:33:29.107538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.107488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerDied","Data":"9741cecde9a510cadf6f6b54b0869332f7152cfb588418751e67104d39fba5b6"}
Aug 04 11:33:29.108987 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.108947    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tbxgw" event={"ID":"2f3f1bb4-c75e-4b61-9db9-034c63028124","Type":"ContainerStarted","Data":"fe8e1e86a64812416dc65385251eb9fd96637d24149ba6ef70438599fa07c2cc"}
Aug 04 11:33:29.114432 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.114395    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-467j9" podStartSLOduration=2.963571142 podStartE2EDuration="22.114383722s" totalImagesPullingTime="19.15081258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:29.114162209 +0000 UTC m=+22.685075012" watchObservedRunningTime="2026-08-04 11:33:29.114383722 +0000 UTC m=+22.685296588"
Aug 04 11:33:29.134326 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.134244    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rk8d8" podStartSLOduration=2.855712973 podStartE2EDuration="22.134229891s" totalImagesPullingTime="19.278516918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:29.134136702 +0000 UTC m=+22.705049507" watchObservedRunningTime="2026-08-04 11:33:29.134229891 +0000 UTC m=+22.705142698"
Aug 04 11:33:29.150477 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.150437    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tbxgw" podStartSLOduration=2.94844954 podStartE2EDuration="22.150421264s" totalImagesPullingTime="19.201971724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:29.149836309 +0000 UTC m=+22.720749114" watchObservedRunningTime="2026-08-04 11:33:29.150421264 +0000 UTC m=+22.721334069"
Aug 04 11:33:29.196736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.196688    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fqkgr" podStartSLOduration=3.027880132 podStartE2EDuration="22.196674419s" totalImagesPullingTime="19.168794287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:29.196261968 +0000 UTC m=+22.767174771" watchObservedRunningTime="2026-08-04 11:33:29.196674419 +0000 UTC m=+22.767587223"
Aug 04 11:33:29.212845 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.212808    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8klxl" podStartSLOduration=3.051513428 podStartE2EDuration="22.212795146s" totalImagesPullingTime="19.161281718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:29.212617355 +0000 UTC m=+22.783530159" watchObservedRunningTime="2026-08-04 11:33:29.212795146 +0000 UTC m=+22.783707949"
Aug 04 11:33:29.911563 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.911534    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 11:33:29.914611 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.914531    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T11:33:29.91155628Z","UUID":"8a25e33d-b720-4738-ab05-1c4cd2a75840","Handler":null,"Name":"","Endpoint":""}
Aug 04 11:33:29.915791 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.915767    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 11:33:29.915873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.915800    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 11:33:29.988380 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:29.988302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:29.988561 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:29.988412    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:30.113302 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:30.113241    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7" event={"ID":"91581844-de78-4503-98e6-fd92d64fa837","Type":"ContainerStarted","Data":"8fbb19603fd4477c0d420624156fd9ca713dc125c3a72158acd1017ef5c3b0a4"}
Aug 04 11:33:30.115013 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:30.114982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5lv9" event={"ID":"dda3ac9d-4ad9-4bef-92fb-190b001daa4e","Type":"ContainerStarted","Data":"6d8143f26084dacaf2ee41b9f1d10fd9b6ed31c3efddadf4bf9a067473b3ad3e"}
Aug 04 11:33:30.131915 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:30.131858    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m5lv9" podStartSLOduration=3.915753771 podStartE2EDuration="23.131839167s" totalImagesPullingTime="19.216085396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:30.131476936 +0000 UTC m=+23.702389741" watchObservedRunningTime="2026-08-04 11:33:30.131839167 +0000 UTC m=+23.702751972"
Aug 04 11:33:30.988699 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:30.988667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:30.989151 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:30.988667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:30.989151 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:30.988809    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:30.989151 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:30.988876    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:31.885088 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:31.884858    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:31.885415 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:31.885391    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:31.987565 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:31.987537    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:31.987711 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:31.987628    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:32.122273 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.122232    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7" event={"ID":"91581844-de78-4503-98e6-fd92d64fa837","Type":"ContainerStarted","Data":"8b880b5dff736da71764176177cdcbe02e8bfc0f34d7ef7c4a7a111796ad9f35"}
Aug 04 11:33:32.129035 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.128960    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"3d049a6fc30374b07536ba0dab3f338ccef51e97084f0d2fa2f8b48504623085"}
Aug 04 11:33:32.129172 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.129106    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:32.129534 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.129450    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-467j9"
Aug 04 11:33:32.144185 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.144106    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd5k7" podStartSLOduration=3.482818504 podStartE2EDuration="25.144093619s" totalImagesPullingTime="21.661275115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:32.143825271 +0000 UTC m=+25.714738075" watchObservedRunningTime="2026-08-04 11:33:32.144093619 +0000 UTC m=+25.715006438"
Aug 04 11:33:32.987963 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.987932    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:32.988149 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:32.988046    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:32.988149 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:32.988086    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:32.988263 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:32.988185    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:33.987595 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:33.987562    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:33.988073 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:33.987680    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:34.987955 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:34.987884    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:34.988474 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:34.987987    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:34.988474 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:34.988079    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:34.988474 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:34.988196    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:35.138700 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.138664    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" event={"ID":"d7d80118-a3de-48a5-b640-5356cc910710","Type":"ContainerStarted","Data":"1bfc1d48f88754927297aee91e3ea657c9a454ac028a26e0ed775daca13900ac"}
Aug 04 11:33:35.138933 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.138905    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:35.140520 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.140488    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f82d5f7-dfb7-4745-8b34-544761a0f69d" containerID="7cf1b6f08b919b92c7dfc7b4b0a40dfec9dda82fedc03be9beaf652ee4c842dc" exitCode=0
Aug 04 11:33:35.140634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.140523    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerDied","Data":"7cf1b6f08b919b92c7dfc7b4b0a40dfec9dda82fedc03be9beaf652ee4c842dc"}
Aug 04 11:33:35.155357 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.155332    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:35.167652 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.167608    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" podStartSLOduration=8.907594337 podStartE2EDuration="28.167592417s" totalImagesPullingTime="19.25999808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:35.166075436 +0000 UTC m=+28.736988241" watchObservedRunningTime="2026-08-04 11:33:35.167592417 +0000 UTC m=+28.738505215"
Aug 04 11:33:35.987661 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:35.987626    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:35.988024 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:35.987758    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:36.142383 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.142352    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 04 11:33:36.142838 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.142805    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:36.157246 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.157224    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:36.546552 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.546472    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gb5hk"]
Aug 04 11:33:36.546693 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.546572    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:36.546693 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:36.546651    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:36.549259 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.549229    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7lbd"]
Aug 04 11:33:36.549404 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.549346    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:36.549462 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:36.549435    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:36.553005 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.552984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j6nsk"]
Aug 04 11:33:36.553093 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:36.553069    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:36.553182 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:36.553154    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:37.148901 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:37.148865    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f82d5f7-dfb7-4745-8b34-544761a0f69d" containerID="2b76258b9bd65f369463acd9fb8cb11f9af28d0828ae664fe659f4261d344226" exitCode=0
Aug 04 11:33:37.149355 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:37.148955    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerDied","Data":"2b76258b9bd65f369463acd9fb8cb11f9af28d0828ae664fe659f4261d344226"}
Aug 04 11:33:37.149355 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:37.149044    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 04 11:33:37.987999 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:37.987966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:37.988181 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:37.988020    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:37.988181 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:37.988092    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:37.988181 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:37.988151    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:37.988181 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:37.988162    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:37.988435 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:37.988229    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:38.153512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:38.153487    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 04 11:33:39.107717 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.107541    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:33:39.121585 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.121537    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" podUID="d7d80118-a3de-48a5-b640-5356cc910710" containerName="ovnkube-controller" probeResult="failure" output=""
Aug 04 11:33:39.129996 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.129961    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" podUID="d7d80118-a3de-48a5-b640-5356cc910710" containerName="ovnkube-controller" probeResult="failure" output=""
Aug 04 11:33:39.157759 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.157736    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f82d5f7-dfb7-4745-8b34-544761a0f69d" containerID="8b85d9aae295481107a9d5b271e663c360610f6c86dc64200553cb69b25d98eb" exitCode=0
Aug 04 11:33:39.158242 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.157823    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerDied","Data":"8b85d9aae295481107a9d5b271e663c360610f6c86dc64200553cb69b25d98eb"}
Aug 04 11:33:39.166880 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.166857    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22" podUID="d7d80118-a3de-48a5-b640-5356cc910710" containerName="ovnkube-controller" probeResult="failure" output=""
Aug 04 11:33:39.988651 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.988617    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:39.988823 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.988714    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:39.988823 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:39.988801    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:39.988823 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:39.988718    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:39.988944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:39.988721    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:39.988944 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:39.988889    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:40.614987 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:40.614951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:40.615559 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.615125    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:40.615559 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.615196    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret podName:7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a nodeName:}" failed. No retries permitted until 2026-08-04 11:34:12.615177481 +0000 UTC m=+66.186090263 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret") pod "global-pull-secret-syncer-j6nsk" (UID: "7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a") : object "kube-system"/"original-pull-secret" not registered
Aug 04 11:33:40.716049 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:40.716017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:40.716209 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.716170    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:40.716264 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.716251    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs podName:cd9918b3-753f-4772-a082-1e922de1494d nodeName:}" failed. No retries permitted until 2026-08-04 11:34:12.716231137 +0000 UTC m=+66.287143933 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs") pod "network-metrics-daemon-s7lbd" (UID: "cd9918b3-753f-4772-a082-1e922de1494d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 11:33:40.817395 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:40.817352    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:40.817647 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.817494    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 11:33:40.817647 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.817508    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 11:33:40.817647 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.817517    2558 projected.go:196] Error preparing data for projected volume kube-api-access-2sxm9 for pod openshift-network-diagnostics/network-check-target-gb5hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:40.817647 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:40.817573    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9 podName:b9c920ad-279e-4d13-a77c-96ddaadbe728 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:12.817558368 +0000 UTC m=+66.388471151 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2sxm9" (UniqueName: "kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9") pod "network-check-target-gb5hk" (UID: "b9c920ad-279e-4d13-a77c-96ddaadbe728") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 11:33:41.988627 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:41.988589    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:41.988627 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:41.988614    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:41.989220 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:41.988621    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:41.989220 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:41.988715    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gb5hk" podUID="b9c920ad-279e-4d13-a77c-96ddaadbe728"
Aug 04 11:33:41.989220 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:41.988810    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j6nsk" podUID="7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a"
Aug 04 11:33:41.989220 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:41.988887    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7lbd" podUID="cd9918b3-753f-4772-a082-1e922de1494d"
Aug 04 11:33:42.207595 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.207513    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-11.ec2.internal" event="NodeReady"
Aug 04 11:33:42.207746 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.207661    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 11:33:42.249014 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.248955    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5fd88f78d6-4h9mv"]
Aug 04 11:33:42.251803 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.251775    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"]
Aug 04 11:33:42.252154 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.252117    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.255313 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.254503    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"]
Aug 04 11:33:42.255313 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.254651    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.255495 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.255474    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 11:33:42.256584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.256564    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"]
Aug 04 11:33:42.256763 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.256749    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.258870    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.259031    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gql25\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.259041    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hdfv9\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.259501    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.259715    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.260131    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.260353    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.261300 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.260671    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-rjvth"]
Aug 04 11:33:42.262131 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.262112    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"
Aug 04 11:33:42.263020 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.262999    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-klg2f"]
Aug 04 11:33:42.263711 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.263328    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.263832 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.263001    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 11:33:42.263918 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.263873    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.264638 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.264620    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.265385 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.265363    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Aug 04 11:33:42.265489 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.265406    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 11:33:42.265489 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.265436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ts7qd\""
Aug 04 11:33:42.265607 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.265547    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.265686 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.265620    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Aug 04 11:33:42.266295 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.266261    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-69g79\""
Aug 04 11:33:42.266651 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.266633    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.266751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.266672    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.266911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.266896    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xlqvl\""
Aug 04 11:33:42.267393 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.267356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.267487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.267470    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Aug 04 11:33:42.269895 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.269829    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.270394 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.270368    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Aug 04 11:33:42.271728 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.271707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Aug 04 11:33:42.272979 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.272905    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.274155 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.274137    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Aug 04 11:33:42.274745 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.274727    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l67rq\""
Aug 04 11:33:42.275068 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.275049    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.276197 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.276179    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"]
Aug 04 11:33:42.281556 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.281537    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tgnj9"]
Aug 04 11:33:42.284398 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.282347    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.285671 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.285326    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Aug 04 11:33:42.287151 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.287051    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.287396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.287355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-v8csx\""
Aug 04 11:33:42.287679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.287658    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"]
Aug 04 11:33:42.287679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.287675    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Aug 04 11:33:42.288075 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.287795    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Aug 04 11:33:42.288075 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.287968    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.289537 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.289508    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.292059 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.291176    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Aug 04 11:33:42.294360 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.293481    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 11:33:42.294360 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.294112    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 11:33:42.294360 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.294331    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hll8p\""
Aug 04 11:33:42.297167 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.295717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"]
Aug 04 11:33:42.297167 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.295749    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"]
Aug 04 11:33:42.297167 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.295781    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6bf9477994-mzx8r"]
Aug 04 11:33:42.297167 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.296034    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:42.298188 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.297982    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"]
Aug 04 11:33:42.300084 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.300050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.301856 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.301649    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-btqpx\""
Aug 04 11:33:42.301856 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.301680    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Aug 04 11:33:42.302336 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.302318    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Aug 04 11:33:42.302703 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.302686    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 04 11:33:42.302922 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.302902    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 04 11:33:42.303168 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.303131    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.303538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.303520    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"]
Aug 04 11:33:42.303703 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.303681    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zhbt4\""
Aug 04 11:33:42.303779 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.303613    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"
Aug 04 11:33:42.304047 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.303578    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 04 11:33:42.304140 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.304116    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 04 11:33:42.304199 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.304180    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.306900 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.306882    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.307315 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.307298    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.307941 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.307920    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:42.308098 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.308079    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nvz4p\""
Aug 04 11:33:42.308301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.307853    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"]
Aug 04 11:33:42.308301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.308214    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-rjvth"]
Aug 04 11:33:42.308301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.308239    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7dznr"]
Aug 04 11:33:42.310566 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.310546    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.311118 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5fd88f78d6-4h9mv"]
Aug 04 11:33:42.311207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"]
Aug 04 11:33:42.311207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311154    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tgnj9"]
Aug 04 11:33:42.311207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311177    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-klg2f"]
Aug 04 11:33:42.311207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311196    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"]
Aug 04 11:33:42.311449 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311224    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7dznr"]
Aug 04 11:33:42.311449 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311239    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"]
Aug 04 11:33:42.311449 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311342    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.311449 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311376    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"]
Aug 04 11:33:42.311969 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.311952    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6bf9477994-mzx8r"]
Aug 04 11:33:42.313178 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.313010    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Aug 04 11:33:42.313178 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.313159    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kkwj7\""
Aug 04 11:33:42.313617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.313571    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.314475 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.313973    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 11:33:42.314475 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.314032    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tcjzq\""
Aug 04 11:33:42.314475 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.314261    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 11:33:42.314475 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.314318    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 11:33:42.428429 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-tmp-dir\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.428429 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.428674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428455    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.428674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428537    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a9605480-7983-409a-b5da-e941c4f7b331-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:42.428674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e45dee60-cff9-46f2-820a-3f7e343524bd-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.428674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428598    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk4px\" (UniqueName: \"kubernetes.io/projected/a6126cb4-a498-49b3-bbcd-0d295304093d-kube-api-access-dk4px\") pod \"volume-data-source-validator-5b8776669-c5vh7\" (UID: \"a6126cb4-a498-49b3-bbcd-0d295304093d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"
Aug 04 11:33:42.428674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-config-volume\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.428674 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-image-registry-private-configuration\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428680    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-serving-cert\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15f98335-5a43-4491-839c-11a71132cb4b-cert\") pod \"ingress-canary-7dznr\" (UID: \"15f98335-5a43-4491-839c-11a71132cb4b\") " pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428764    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a86e8b9-802f-442f-b6e8-4daf7b39d025-serving-cert\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428785    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428812    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9xm\" (UniqueName: \"kubernetes.io/projected/f5ad681e-ef93-4950-93de-047bab90b171-kube-api-access-gn9xm\") pod \"network-check-source-65bdbbff45-p7bpm\" (UID: \"f5ad681e-ef93-4950-93de-047bab90b171\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33a40371-2939-448c-a467-d338dad327d1-tmp\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428886    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33a40371-2939-448c-a467-d338dad327d1-trusted-ca-bundle\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-stats-auth\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428918    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p88zq\" (UniqueName: \"kubernetes.io/projected/5a86e8b9-802f-442f-b6e8-4daf7b39d025-kube-api-access-p88zq\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.428950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ms9\" (UniqueName: \"kubernetes.io/projected/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-kube-api-access-v4ms9\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.428976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e45dee60-cff9-46f2-820a-3f7e343524bd-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429020    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429067    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmnhc\" (UniqueName: \"kubernetes.io/projected/26e8828f-fa45-4972-8389-98db03ce35f3-kube-api-access-jmnhc\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429144    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a86e8b9-802f-442f-b6e8-4daf7b39d025-config\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429168    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5946213e-2cba-468c-9683-fe335d33ba70-ca-trust-extracted\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-bound-sa-token\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/33a40371-2939-448c-a467-d338dad327d1-snapshots\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429246    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tqpl\" (UniqueName: \"kubernetes.io/projected/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-kube-api-access-7tqpl\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429275    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-installation-pull-secrets\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429319    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf2bd\" (UniqueName: \"kubernetes.io/projected/33a40371-2939-448c-a467-d338dad327d1-kube-api-access-bf2bd\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429346    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-registry-certificates\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqmg\" (UniqueName: \"kubernetes.io/projected/e45dee60-cff9-46f2-820a-3f7e343524bd-kube-api-access-mvqmg\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.429561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429393    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zpbh\" (UniqueName: \"kubernetes.io/projected/55b006b1-9d28-428c-89ec-07c23c853b5b-kube-api-access-9zpbh\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429455    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-trusted-ca\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429512    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33a40371-2939-448c-a467-d338dad327d1-service-ca-bundle\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429545    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4880a511-b249-4367-9811-55fcbc037524-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq67r\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-kube-api-access-fq67r\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-default-certificate\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a86e8b9-802f-442f-b6e8-4daf7b39d025-trusted-ca\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429646    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqt65\" (UniqueName: \"kubernetes.io/projected/4880a511-b249-4367-9811-55fcbc037524-kube-api-access-tqt65\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429669    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33a40371-2939-448c-a467-d338dad327d1-serving-cert\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429703    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-config\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.430136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.429747    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ckxg\" (UniqueName: \"kubernetes.io/projected/15f98335-5a43-4491-839c-11a71132cb4b-kube-api-access-6ckxg\") pod \"ingress-canary-7dznr\" (UID: \"15f98335-5a43-4491-839c-11a71132cb4b\") " pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.530304 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-config\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530314    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ckxg\" (UniqueName: \"kubernetes.io/projected/15f98335-5a43-4491-839c-11a71132cb4b-kube-api-access-6ckxg\") pod \"ingress-canary-7dznr\" (UID: \"15f98335-5a43-4491-839c-11a71132cb4b\") " pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-tmp-dir\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530366    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530424    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a9605480-7983-409a-b5da-e941c4f7b331-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530445    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e45dee60-cff9-46f2-820a-3f7e343524bd-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530466    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4px\" (UniqueName: \"kubernetes.io/projected/a6126cb4-a498-49b3-bbcd-0d295304093d-kube-api-access-dk4px\") pod \"volume-data-source-validator-5b8776669-c5vh7\" (UID: \"a6126cb4-a498-49b3-bbcd-0d295304093d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"
Aug 04 11:33:42.530487 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530488    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-config-volume\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-image-registry-private-configuration\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.530536    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.030513172 +0000 UTC m=+36.601425969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : configmap references non-existent config key: service-ca.crt
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-serving-cert\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15f98335-5a43-4491-839c-11a71132cb4b-cert\") pod \"ingress-canary-7dznr\" (UID: \"15f98335-5a43-4491-839c-11a71132cb4b\") " pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a86e8b9-802f-442f-b6e8-4daf7b39d025-serving-cert\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530650    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9xm\" (UniqueName: \"kubernetes.io/projected/f5ad681e-ef93-4950-93de-047bab90b171-kube-api-access-gn9xm\") pod \"network-check-source-65bdbbff45-p7bpm\" (UID: \"f5ad681e-ef93-4950-93de-047bab90b171\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530668    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-tmp-dir\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33a40371-2939-448c-a467-d338dad327d1-tmp\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33a40371-2939-448c-a467-d338dad327d1-trusted-ca-bundle\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-stats-auth\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530735    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p88zq\" (UniqueName: \"kubernetes.io/projected/5a86e8b9-802f-442f-b6e8-4daf7b39d025-kube-api-access-p88zq\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ms9\" (UniqueName: \"kubernetes.io/projected/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-kube-api-access-v4ms9\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.530759    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530767    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e45dee60-cff9-46f2-820a-3f7e343524bd-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.530873 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.530774    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd88f78d6-4h9mv: secret "image-registry-tls" not found
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.530834    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls podName:5946213e-2cba-468c-9683-fe335d33ba70 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.030819349 +0000 UTC m=+36.601732149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls") pod "image-registry-5fd88f78d6-4h9mv" (UID: "5946213e-2cba-468c-9683-fe335d33ba70") : secret "image-registry-tls" not found
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.530935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-config\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmnhc\" (UniqueName: \"kubernetes.io/projected/26e8828f-fa45-4972-8389-98db03ce35f3-kube-api-access-jmnhc\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a86e8b9-802f-442f-b6e8-4daf7b39d025-config\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531380    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e45dee60-cff9-46f2-820a-3f7e343524bd-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.531453    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:42.531604 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.531497    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls podName:4880a511-b249-4367-9811-55fcbc037524 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.031481967 +0000 UTC m=+36.602394760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-dsb2c" (UID: "4880a511-b249-4367-9811-55fcbc037524") : secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:42.531986 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531776    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5946213e-2cba-468c-9683-fe335d33ba70-ca-trust-extracted\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.531986 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.531897    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 11:33:42.532076 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.532001    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.031980032 +0000 UTC m=+36.602892819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : secret "router-metrics-certs-default" not found
Aug 04 11:33:42.532127 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a9605480-7983-409a-b5da-e941c4f7b331-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:42.532174 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.531382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5946213e-2cba-468c-9683-fe335d33ba70-ca-trust-extracted\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.532221 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-bound-sa-token\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.532270 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532231    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/33a40371-2939-448c-a467-d338dad327d1-snapshots\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.532270 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532257    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tqpl\" (UniqueName: \"kubernetes.io/projected/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-kube-api-access-7tqpl\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.532396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-installation-pull-secrets\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.532396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532329    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf2bd\" (UniqueName: \"kubernetes.io/projected/33a40371-2939-448c-a467-d338dad327d1-kube-api-access-bf2bd\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.532396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-registry-certificates\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.532396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqmg\" (UniqueName: \"kubernetes.io/projected/e45dee60-cff9-46f2-820a-3f7e343524bd-kube-api-access-mvqmg\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.532590 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532416    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:42.532590 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zpbh\" (UniqueName: \"kubernetes.io/projected/55b006b1-9d28-428c-89ec-07c23c853b5b-kube-api-access-9zpbh\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:42.532590 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-trusted-ca\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.532590 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33a40371-2939-448c-a467-d338dad327d1-service-ca-bundle\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.532590 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4880a511-b249-4367-9811-55fcbc037524-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.532853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq67r\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-kube-api-access-fq67r\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.532853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-default-certificate\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.532853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a86e8b9-802f-442f-b6e8-4daf7b39d025-trusted-ca\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.532853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532695    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqt65\" (UniqueName: \"kubernetes.io/projected/4880a511-b249-4367-9811-55fcbc037524-kube-api-access-tqt65\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.532853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532722    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33a40371-2939-448c-a467-d338dad327d1-serving-cert\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.532853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.532817    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a86e8b9-802f-442f-b6e8-4daf7b39d025-config\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.533136 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.532901    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 11:33:42.533136 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.532952    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls podName:55b006b1-9d28-428c-89ec-07c23c853b5b nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.032930433 +0000 UTC m=+36.603843231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-pbmmv" (UID: "55b006b1-9d28-428c-89ec-07c23c853b5b") : secret "samples-operator-tls" not found
Aug 04 11:33:42.533136 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.531453    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 11:33:42.533136 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.532997    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert podName:a9605480-7983-409a-b5da-e941c4f7b331 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.03298626 +0000 UTC m=+36.603899057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b2z74" (UID: "a9605480-7983-409a-b5da-e941c4f7b331") : secret "networking-console-plugin-cert" not found
Aug 04 11:33:42.533706 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.533681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/33a40371-2939-448c-a467-d338dad327d1-snapshots\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.533802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.533729    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33a40371-2939-448c-a467-d338dad327d1-service-ca-bundle\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.533861 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.533823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4880a511-b249-4367-9811-55fcbc037524-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.534133 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.534106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-trusted-ca\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.534427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.534405    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33a40371-2939-448c-a467-d338dad327d1-tmp\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.535047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-image-registry-private-configuration\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.535066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15f98335-5a43-4491-839c-11a71132cb4b-cert\") pod \"ingress-canary-7dznr\" (UID: \"15f98335-5a43-4491-839c-11a71132cb4b\") " pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.535609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-config-volume\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.536435    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-serving-cert\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.536741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33a40371-2939-448c-a467-d338dad327d1-trusted-ca-bundle\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.537186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-registry-certificates\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.537295    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:42.537348    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls podName:2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba nodeName:}" failed. No retries permitted until 2026-08-04 11:33:43.037333081 +0000 UTC m=+36.608245874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls") pod "dns-default-tgnj9" (UID: "2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba") : secret "dns-default-metrics-tls" not found
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.537379    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a86e8b9-802f-442f-b6e8-4daf7b39d025-trusted-ca\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.537714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a86e8b9-802f-442f-b6e8-4daf7b39d025-serving-cert\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.537898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-default-certificate\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.537901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e45dee60-cff9-46f2-820a-3f7e343524bd-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.538064    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-stats-auth\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.538203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.538170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33a40371-2939-448c-a467-d338dad327d1-serving-cert\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.538989 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.538795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-installation-pull-secrets\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.540179 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.540135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ckxg\" (UniqueName: \"kubernetes.io/projected/15f98335-5a43-4491-839c-11a71132cb4b-kube-api-access-6ckxg\") pod \"ingress-canary-7dznr\" (UID: \"15f98335-5a43-4491-839c-11a71132cb4b\") " pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:42.540717 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.540694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk4px\" (UniqueName: \"kubernetes.io/projected/a6126cb4-a498-49b3-bbcd-0d295304093d-kube-api-access-dk4px\") pod \"volume-data-source-validator-5b8776669-c5vh7\" (UID: \"a6126cb4-a498-49b3-bbcd-0d295304093d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"
Aug 04 11:33:42.541520 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.541446    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmnhc\" (UniqueName: \"kubernetes.io/projected/26e8828f-fa45-4972-8389-98db03ce35f3-kube-api-access-jmnhc\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:42.543060 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.543034    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tqpl\" (UniqueName: \"kubernetes.io/projected/238cdeaf-02dc-45e2-94d5-0fb95fe656a9-kube-api-access-7tqpl\") pod \"service-ca-operator-784f97bcf6-8xpt9\" (UID: \"238cdeaf-02dc-45e2-94d5-0fb95fe656a9\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.543153 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.543066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-bound-sa-token\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.543222 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.543145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zpbh\" (UniqueName: \"kubernetes.io/projected/55b006b1-9d28-428c-89ec-07c23c853b5b-kube-api-access-9zpbh\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:42.544865 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.544834    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p88zq\" (UniqueName: \"kubernetes.io/projected/5a86e8b9-802f-442f-b6e8-4daf7b39d025-kube-api-access-p88zq\") pod \"console-operator-5ccd6b445f-rjvth\" (UID: \"5a86e8b9-802f-442f-b6e8-4daf7b39d025\") " pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.544865 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.544851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqt65\" (UniqueName: \"kubernetes.io/projected/4880a511-b249-4367-9811-55fcbc037524-kube-api-access-tqt65\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:42.547920 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.547901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf2bd\" (UniqueName: \"kubernetes.io/projected/33a40371-2939-448c-a467-d338dad327d1-kube-api-access-bf2bd\") pod \"insights-operator-77696685b-klg2f\" (UID: \"33a40371-2939-448c-a467-d338dad327d1\") " pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.550054 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.550026    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ms9\" (UniqueName: \"kubernetes.io/projected/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-kube-api-access-v4ms9\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:42.550305 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.550270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9xm\" (UniqueName: \"kubernetes.io/projected/f5ad681e-ef93-4950-93de-047bab90b171-kube-api-access-gn9xm\") pod \"network-check-source-65bdbbff45-p7bpm\" (UID: \"f5ad681e-ef93-4950-93de-047bab90b171\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"
Aug 04 11:33:42.550418 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.550398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqmg\" (UniqueName: \"kubernetes.io/projected/e45dee60-cff9-46f2-820a-3f7e343524bd-kube-api-access-mvqmg\") pod \"kube-storage-version-migrator-operator-7878f4b88c-5jqsp\" (UID: \"e45dee60-cff9-46f2-820a-3f7e343524bd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.550535 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.550416    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq67r\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-kube-api-access-fq67r\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:42.577197 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.577175    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"
Aug 04 11:33:42.605272 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.605233    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"
Aug 04 11:33:42.613386 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.613365    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:42.619994 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.619977    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-klg2f"
Aug 04 11:33:42.627021 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.626998    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"
Aug 04 11:33:42.656062 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.656039    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"
Aug 04 11:33:42.669817 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:42.669800    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7dznr"
Aug 04 11:33:43.037561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:43.037561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037566    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037612    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037688    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037749    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert podName:a9605480-7983-409a-b5da-e941c4f7b331 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.037731374 +0000 UTC m=+37.608644157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b2z74" (UID: "a9605480-7983-409a-b5da-e941c4f7b331") : secret "networking-console-plugin-cert" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037767    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.037757489 +0000 UTC m=+37.608670272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : configmap references non-existent config key: service-ca.crt
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037767    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037822    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls podName:55b006b1-9d28-428c-89ec-07c23c853b5b nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.037810409 +0000 UTC m=+37.608723190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-pbmmv" (UID: "55b006b1-9d28-428c-89ec-07c23c853b5b") : secret "samples-operator-tls" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037771    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037873    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037899    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.037884464 +0000 UTC m=+37.608797258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : secret "router-metrics-certs-default" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037920    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls podName:4880a511-b249-4367-9811-55fcbc037524 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.037911212 +0000 UTC m=+37.608824011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-dsb2c" (UID: "4880a511-b249-4367-9811-55fcbc037524") : secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037949    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 11:33:43.038143 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.037952    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:43.038676 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037963    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd88f78d6-4h9mv: secret "image-registry-tls" not found
Aug 04 11:33:43.038676 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.037993    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls podName:5946213e-2cba-468c-9683-fe335d33ba70 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.037984031 +0000 UTC m=+37.608896813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls") pod "image-registry-5fd88f78d6-4h9mv" (UID: "5946213e-2cba-468c-9683-fe335d33ba70") : secret "image-registry-tls" not found
Aug 04 11:33:43.038676 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.038026    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 11:33:43.038676 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:43.038065    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls podName:2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.038051934 +0000 UTC m=+37.608964722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls") pod "dns-default-tgnj9" (UID: "2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba") : secret "dns-default-metrics-tls" not found
Aug 04 11:33:43.710871 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.710591    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-rjvth"]
Aug 04 11:33:43.712074 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.712049    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7"]
Aug 04 11:33:43.717125 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.717052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp"]
Aug 04 11:33:43.732739 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.732714    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm"]
Aug 04 11:33:43.735938 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.735889    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9"]
Aug 04 11:33:43.736905 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.736875    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-klg2f"]
Aug 04 11:33:43.747384 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.747352    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7dznr"]
Aug 04 11:33:43.750610 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:43.750581    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238cdeaf_02dc_45e2_94d5_0fb95fe656a9.slice/crio-9fe0983602ddfd759ec697fcf4c97efa1313679111fcd6bb6b75b48915fa0b10 WatchSource:0}: Error finding container 9fe0983602ddfd759ec697fcf4c97efa1313679111fcd6bb6b75b48915fa0b10: Status 404 returned error can't find the container with id 9fe0983602ddfd759ec697fcf4c97efa1313679111fcd6bb6b75b48915fa0b10
Aug 04 11:33:43.751371 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:43.751347    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ad681e_ef93_4950_93de_047bab90b171.slice/crio-c613d8b53a6f4f92b5d7cb02e9107c7a92cfca0c0ad87fa832c83274299112ac WatchSource:0}: Error finding container c613d8b53a6f4f92b5d7cb02e9107c7a92cfca0c0ad87fa832c83274299112ac: Status 404 returned error can't find the container with id c613d8b53a6f4f92b5d7cb02e9107c7a92cfca0c0ad87fa832c83274299112ac
Aug 04 11:33:43.753025 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:33:43.753003    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f98335_5a43_4491_839c_11a71132cb4b.slice/crio-332f6b8632cbd4245f32effa412cf6c95ec73af2270509b392399bfdb6c51149 WatchSource:0}: Error finding container 332f6b8632cbd4245f32effa412cf6c95ec73af2270509b392399bfdb6c51149: Status 404 returned error can't find the container with id 332f6b8632cbd4245f32effa412cf6c95ec73af2270509b392399bfdb6c51149
Aug 04 11:33:43.988555 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.988458    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:33:43.988719 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.988585    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:33:43.988719 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.988629    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:33:43.991352 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.991328    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 11:33:43.991462 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.991372    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9rm6g\""
Aug 04 11:33:43.991504 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.991334    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 11:33:43.991549 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:43.991332    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zp2md\""
Aug 04 11:33:44.048666 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.048643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.048694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.048712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048790    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048809    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048844    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls podName:55b006b1-9d28-428c-89ec-07c23c853b5b nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.048825526 +0000 UTC m=+39.619738325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-pbmmv" (UID: "55b006b1-9d28-428c-89ec-07c23c853b5b") : secret "samples-operator-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048859    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.048853263 +0000 UTC m=+39.619766044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : configmap references non-existent config key: service-ca.crt
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.048887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048925    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls podName:4880a511-b249-4367-9811-55fcbc037524 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.048899363 +0000 UTC m=+39.619812159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-dsb2c" (UID: "4880a511-b249-4367-9811-55fcbc037524") : secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048937    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048946    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd88f78d6-4h9mv: secret "image-registry-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.048975    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls podName:5946213e-2cba-468c-9683-fe335d33ba70 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.048966316 +0000 UTC m=+39.619879121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls") pod "image-registry-5fd88f78d6-4h9mv" (UID: "5946213e-2cba-468c-9683-fe335d33ba70") : secret "image-registry-tls" not found
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.048973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.049040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:44.049071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.049069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:44.049634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.049077    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 11:33:44.049634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.049089    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 11:33:44.049634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.049165    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls podName:2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.049151221 +0000 UTC m=+39.620064022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls") pod "dns-default-tgnj9" (UID: "2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba") : secret "dns-default-metrics-tls" not found
Aug 04 11:33:44.049634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.049106    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 11:33:44.049634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.049184    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert podName:a9605480-7983-409a-b5da-e941c4f7b331 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.049177359 +0000 UTC m=+39.620090140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b2z74" (UID: "a9605480-7983-409a-b5da-e941c4f7b331") : secret "networking-console-plugin-cert" not found
Aug 04 11:33:44.049634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:44.049206    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:46.049197032 +0000 UTC m=+39.620109830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : secret "router-metrics-certs-default" not found
Aug 04 11:33:44.170150 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.170117    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7" event={"ID":"a6126cb4-a498-49b3-bbcd-0d295304093d","Type":"ContainerStarted","Data":"f876ff9c35448c60b6d1bac4c9aa3494c50fc52a9673fbcab69a26fb8e546bbd"}
Aug 04 11:33:44.171341 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.171314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7dznr" event={"ID":"15f98335-5a43-4491-839c-11a71132cb4b","Type":"ContainerStarted","Data":"332f6b8632cbd4245f32effa412cf6c95ec73af2270509b392399bfdb6c51149"}
Aug 04 11:33:44.172519 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.172494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9" event={"ID":"238cdeaf-02dc-45e2-94d5-0fb95fe656a9","Type":"ContainerStarted","Data":"9fe0983602ddfd759ec697fcf4c97efa1313679111fcd6bb6b75b48915fa0b10"}
Aug 04 11:33:44.173692 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.173667    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm" event={"ID":"f5ad681e-ef93-4950-93de-047bab90b171","Type":"ContainerStarted","Data":"c613d8b53a6f4f92b5d7cb02e9107c7a92cfca0c0ad87fa832c83274299112ac"}
Aug 04 11:33:44.174953 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.174930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp" event={"ID":"e45dee60-cff9-46f2-820a-3f7e343524bd","Type":"ContainerStarted","Data":"d9d0da769fc164b9efc1ceb1859d0fe71228fe2580c253d63f79e6c336040b96"}
Aug 04 11:33:44.178318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.178294    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f82d5f7-dfb7-4745-8b34-544761a0f69d" containerID="967d1811d94b583e7791ab0ce2bbb335b1640e9185cb218fdc4d3f5e6d238e15" exitCode=0
Aug 04 11:33:44.178423 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.178368    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerDied","Data":"967d1811d94b583e7791ab0ce2bbb335b1640e9185cb218fdc4d3f5e6d238e15"}
Aug 04 11:33:44.179868 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.179843    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" event={"ID":"5a86e8b9-802f-442f-b6e8-4daf7b39d025","Type":"ContainerStarted","Data":"a6292abcc57774b67f1b4044450aa5bd4f01ac23470c2ebe72247e19e70e6094"}
Aug 04 11:33:44.181153 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:44.181132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-klg2f" event={"ID":"33a40371-2939-448c-a467-d338dad327d1","Type":"ContainerStarted","Data":"416bd151ed4e9cece7d30bc7d8d8de0161a67e5d1932b9d5f8959d3b2c4bb091"}
Aug 04 11:33:45.203881 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:45.202690    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f82d5f7-dfb7-4745-8b34-544761a0f69d" containerID="80d1b73fe9d167f84c2adab9eb549a941d98fe0792151ef52722b92f3c909952" exitCode=0
Aug 04 11:33:45.203881 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:45.202972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerDied","Data":"80d1b73fe9d167f84c2adab9eb549a941d98fe0792151ef52722b92f3c909952"}
Aug 04 11:33:45.513563 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:45.513472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:45.603234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:45.601983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:46.075223 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:46.075223 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:46.075452 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:46.075452 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075341    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075317909 +0000 UTC m=+43.646230694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : configmap references non-existent config key: service-ca.crt
Aug 04 11:33:46.075452 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075398    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 11:33:46.075452 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075414    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd88f78d6-4h9mv: secret "image-registry-tls" not found
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075473    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls podName:5946213e-2cba-468c-9683-fe335d33ba70 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075457728 +0000 UTC m=+43.646370533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls") pod "image-registry-5fd88f78d6-4h9mv" (UID: "5946213e-2cba-468c-9683-fe335d33ba70") : secret "image-registry-tls" not found
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075525    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075568    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls podName:4880a511-b249-4367-9811-55fcbc037524 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075555517 +0000 UTC m=+43.646468323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-dsb2c" (UID: "4880a511-b249-4367-9811-55fcbc037524") : secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075623    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075644    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:46.075651 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075653    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls podName:2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075642747 +0000 UTC m=+43.646555543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls") pod "dns-default-tgnj9" (UID: "2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba") : secret "dns-default-metrics-tls" not found
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075690    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075706    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075735    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert podName:a9605480-7983-409a-b5da-e941c4f7b331 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075725867 +0000 UTC m=+43.646638656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b2z74" (UID: "a9605480-7983-409a-b5da-e941c4f7b331") : secret "networking-console-plugin-cert" not found
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.075765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075775    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075806    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075795882 +0000 UTC m=+43.646708677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : secret "router-metrics-certs-default" not found
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075839    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 11:33:46.076014 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:46.075875    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls podName:55b006b1-9d28-428c-89ec-07c23c853b5b nodeName:}" failed. No retries permitted until 2026-08-04 11:33:50.075864423 +0000 UTC m=+43.646777218 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-pbmmv" (UID: "55b006b1-9d28-428c-89ec-07c23c853b5b") : secret "samples-operator-tls" not found
Aug 04 11:33:46.212649 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.212611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" event={"ID":"9f82d5f7-dfb7-4745-8b34-544761a0f69d","Type":"ContainerStarted","Data":"82f994b9f8d8df3e5be8117dd42fa805c95217d0515bdb6c553c9bb27671d2b8"}
Aug 04 11:33:46.214749 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.214721    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7" event={"ID":"a6126cb4-a498-49b3-bbcd-0d295304093d","Type":"ContainerStarted","Data":"5620e1e7b1fed9cb5a4e481c10dafe54b5505354bbcb7645a57ec3facd467b4e"}
Aug 04 11:33:46.259867 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.259809    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c8pjg" podStartSLOduration=8.159735672 podStartE2EDuration="39.259793548s" totalImagesPullingTime="31.100057876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:46.242887477 +0000 UTC m=+39.813800284" watchObservedRunningTime="2026-08-04 11:33:46.259793548 +0000 UTC m=+39.830706352"
Aug 04 11:33:46.261070 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.261016    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-c5vh7" podStartSLOduration=17.195128848 podStartE2EDuration="19.261003596s" totalImagesPullingTime="2.065874748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:46.25970572 +0000 UTC m=+39.830618524" watchObservedRunningTime="2026-08-04 11:33:46.261003596 +0000 UTC m=+39.831916400"
Aug 04 11:33:46.364797 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.364707    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:46.465888 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.465817    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:46.570372 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.570336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:46.646453 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:46.646351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:48.176463 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:48.176420    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:48.232472 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:48.232436    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:50.114176 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114260    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114318    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114377    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114354441 +0000 UTC m=+51.685267227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : configmap references non-existent config key: service-ca.crt
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114395    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114438    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114451    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd88f78d6-4h9mv: secret "image-registry-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114439    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls podName:4880a511-b249-4367-9811-55fcbc037524 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114423102 +0000 UTC m=+51.685335886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-dsb2c" (UID: "4880a511-b249-4367-9811-55fcbc037524") : secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114488    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls podName:55b006b1-9d28-428c-89ec-07c23c853b5b nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114469347 +0000 UTC m=+51.685382143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-pbmmv" (UID: "55b006b1-9d28-428c-89ec-07c23c853b5b") : secret "samples-operator-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114511    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114514    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls podName:5946213e-2cba-468c-9683-fe335d33ba70 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114501751 +0000 UTC m=+51.685414533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls") pod "image-registry-5fd88f78d6-4h9mv" (UID: "5946213e-2cba-468c-9683-fe335d33ba70") : secret "image-registry-tls" not found
Aug 04 11:33:50.114608 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114568    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls podName:2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114553691 +0000 UTC m=+51.685466487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls") pod "dns-default-tgnj9" (UID: "2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba") : secret "dns-default-metrics-tls" not found
Aug 04 11:33:50.115207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114627    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:50.115207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:50.114655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:50.115207 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114755    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 11:33:50.115207 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114827    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114813284 +0000 UTC m=+51.685726069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : secret "router-metrics-certs-default" not found
Aug 04 11:33:50.115207 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114862    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 11:33:50.115207 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:50.114903    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert podName:a9605480-7983-409a-b5da-e941c4f7b331 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:58.114890275 +0000 UTC m=+51.685803083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b2z74" (UID: "a9605480-7983-409a-b5da-e941c4f7b331") : secret "networking-console-plugin-cert" not found
Aug 04 11:33:51.423869 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:51.423778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:51.524006 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:51.523924    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:51.791180 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:51.791150    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:51.855676 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:51.855641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:52.232247 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.232171    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/0.log"
Aug 04 11:33:52.232247 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.232213    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a86e8b9-802f-442f-b6e8-4daf7b39d025" containerID="b77f8f8c2f33d1c7173dd03e19afb9da8b934c8e66bf4c7f48ee4a32f13f9821" exitCode=255
Aug 04 11:33:52.232447 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.232315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" event={"ID":"5a86e8b9-802f-442f-b6e8-4daf7b39d025","Type":"ContainerDied","Data":"b77f8f8c2f33d1c7173dd03e19afb9da8b934c8e66bf4c7f48ee4a32f13f9821"}
Aug 04 11:33:52.232688 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.232668    2558 scope.go:117] "RemoveContainer" containerID="b77f8f8c2f33d1c7173dd03e19afb9da8b934c8e66bf4c7f48ee4a32f13f9821"
Aug 04 11:33:52.235043 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.234969    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-klg2f" event={"ID":"33a40371-2939-448c-a467-d338dad327d1","Type":"ContainerStarted","Data":"82ac8cfdbe0f092c75bf0cd9cb9d9e5dddff7a6257bd49b6c11aeb072e2c710c"}
Aug 04 11:33:52.236992 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.236966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7dznr" event={"ID":"15f98335-5a43-4491-839c-11a71132cb4b","Type":"ContainerStarted","Data":"cd68ed632db4f28c472d541554e702cf589b1243ff8c04655b88700c8cea3864"}
Aug 04 11:33:52.238835 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.238813    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9" event={"ID":"238cdeaf-02dc-45e2-94d5-0fb95fe656a9","Type":"ContainerStarted","Data":"1690902e6b2ca052edd874d0a352395627d4554ace60763a423cdff695b13425"}
Aug 04 11:33:52.240382 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.240361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm" event={"ID":"f5ad681e-ef93-4950-93de-047bab90b171","Type":"ContainerStarted","Data":"b276be884e2df9eff6cebbe635596415b094c1189b346451ced03cfb83b7f001"}
Aug 04 11:33:52.241862 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.241826    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp" event={"ID":"e45dee60-cff9-46f2-820a-3f7e343524bd","Type":"ContainerStarted","Data":"fa246a5436c3f314e4b8506fe4c1c839b213b8792d134d221cab3616f46f7326"}
Aug 04 11:33:52.265396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.265349    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp" podStartSLOduration=17.730781065 podStartE2EDuration="25.265336599s" totalImagesPullingTime="7.534555534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:52.264527329 +0000 UTC m=+45.835440138" watchObservedRunningTime="2026-08-04 11:33:52.265336599 +0000 UTC m=+45.836249403"
Aug 04 11:33:52.281984 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.281941    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-klg2f" podStartSLOduration=17.781738801 podStartE2EDuration="25.281923531s" totalImagesPullingTime="7.50018473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:52.281632721 +0000 UTC m=+45.852545527" watchObservedRunningTime="2026-08-04 11:33:52.281923531 +0000 UTC m=+45.852836341"
Aug 04 11:33:52.302068 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.302020    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7dznr" podStartSLOduration=2.802026296 podStartE2EDuration="10.301986437s" totalImagesPullingTime="7.499960141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:52.300132061 +0000 UTC m=+45.871044866" watchObservedRunningTime="2026-08-04 11:33:52.301986437 +0000 UTC m=+45.872899239"
Aug 04 11:33:52.316692 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.316643    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-p7bpm" podStartSLOduration=17.800813788 podStartE2EDuration="25.316625094s" totalImagesPullingTime="7.515811306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:52.315991796 +0000 UTC m=+45.886904604" watchObservedRunningTime="2026-08-04 11:33:52.316625094 +0000 UTC m=+45.887537901"
Aug 04 11:33:52.335711 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.335660    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9" podStartSLOduration=17.83406759 podStartE2EDuration="25.335640762s" totalImagesPullingTime="7.501573172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:52.335233246 +0000 UTC m=+45.906146054" watchObservedRunningTime="2026-08-04 11:33:52.335640762 +0000 UTC m=+45.906553566"
Aug 04 11:33:52.614461 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.614424    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:52.614911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.614517    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:33:52.916413 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.916324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:52.916899 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.916879    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7k5jv"]
Aug 04 11:33:52.945186 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.945148    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7k5jv"]
Aug 04 11:33:52.945403 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.945390    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:52.948572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.948325    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 11:33:52.948572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.948367    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fqqnm\""
Aug 04 11:33:52.948756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.948598    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 11:33:52.989291 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:52.989260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:53.039492 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.039462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2c90932-c01e-4d72-ba89-ebce20441511-data-volume\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.039669 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.039524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.039669 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.039594    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.039759 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.039662    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d2c90932-c01e-4d72-ba89-ebce20441511-crio-socket\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.039759 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.039689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmp78\" (UniqueName: \"kubernetes.io/projected/d2c90932-c01e-4d72-ba89-ebce20441511-kube-api-access-dmp78\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.140893 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.140851    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2c90932-c01e-4d72-ba89-ebce20441511-data-volume\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.141070 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.140919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.141070 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.140959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.141070 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.141057    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 11:33:53.141243 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.141100    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:53.141243 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.141112    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:53.641094244 +0000 UTC m=+47.212007031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : configmap "kube-rbac-proxy" not found
Aug 04 11:33:53.141243 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.141170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d2c90932-c01e-4d72-ba89-ebce20441511-crio-socket\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.141243 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.141210    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:53.641190631 +0000 UTC m=+47.212103426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:53.141243 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.141234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d2c90932-c01e-4d72-ba89-ebce20441511-crio-socket\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.141479 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.141238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2c90932-c01e-4d72-ba89-ebce20441511-data-volume\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.141479 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.141269    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmp78\" (UniqueName: \"kubernetes.io/projected/d2c90932-c01e-4d72-ba89-ebce20441511-kube-api-access-dmp78\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.167537 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.167474    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmp78\" (UniqueName: \"kubernetes.io/projected/d2c90932-c01e-4d72-ba89-ebce20441511-kube-api-access-dmp78\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.194861 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.194824    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:53.249698 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.249668    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/1.log"
Aug 04 11:33:53.250161 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.250145    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/0.log"
Aug 04 11:33:53.250247 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.250181    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a86e8b9-802f-442f-b6e8-4daf7b39d025" containerID="931b46a3b83cc136fe2db23db63bd5beedc99735e978158820b7edbb6db6c3cd" exitCode=255
Aug 04 11:33:53.250332 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.250306    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" event={"ID":"5a86e8b9-802f-442f-b6e8-4daf7b39d025","Type":"ContainerDied","Data":"931b46a3b83cc136fe2db23db63bd5beedc99735e978158820b7edbb6db6c3cd"}
Aug 04 11:33:53.250405 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.250390    2558 scope.go:117] "RemoveContainer" containerID="b77f8f8c2f33d1c7173dd03e19afb9da8b934c8e66bf4c7f48ee4a32f13f9821"
Aug 04 11:33:53.250640 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.250616    2558 scope.go:117] "RemoveContainer" containerID="931b46a3b83cc136fe2db23db63bd5beedc99735e978158820b7edbb6db6c3cd"
Aug 04 11:33:53.250842 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.250801    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-rjvth_openshift-console-operator(5a86e8b9-802f-442f-b6e8-4daf7b39d025)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" podUID="5a86e8b9-802f-442f-b6e8-4daf7b39d025"
Aug 04 11:33:53.284620 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.284573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:53.406996 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.406966    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:53.462974 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.462903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:53.646161 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.646119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.646161 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.646168    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:53.646634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.646249    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 11:33:53.646634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.646271    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:53.646634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.646326    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:54.646312454 +0000 UTC m=+48.217225236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : configmap "kube-rbac-proxy" not found
Aug 04 11:33:53.646634 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:53.646339    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:54.646333115 +0000 UTC m=+48.217245898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:53.839988 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.839952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:53.941181 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:53.941149    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:54.114639 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.114563    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:54.188684 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.188651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:54.254438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.254412    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/1.log"
Aug 04 11:33:54.254767 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.254753    2558 scope.go:117] "RemoveContainer" containerID="931b46a3b83cc136fe2db23db63bd5beedc99735e978158820b7edbb6db6c3cd"
Aug 04 11:33:54.254926 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:54.254907    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-rjvth_openshift-console-operator(5a86e8b9-802f-442f-b6e8-4daf7b39d025)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" podUID="5a86e8b9-802f-442f-b6e8-4daf7b39d025"
Aug 04 11:33:54.261054 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.261032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:54.655220 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.655190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:54.655679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.655231    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:54.655679 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:54.655364    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:54.655679 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:54.655403    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 11:33:54.655679 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:54.655428    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:56.655413998 +0000 UTC m=+50.226326779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:54.655679 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:54.655500    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:56.655481073 +0000 UTC m=+50.226393873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : configmap "kube-rbac-proxy" not found
Aug 04 11:33:54.676729 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:54.676711    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8klxl_34457f78-d6ea-4f3b-9061-19bc08898912/dns-node-resolver/0.log"
Aug 04 11:33:55.757324 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:55.757292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:55.907154 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:55.907125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:56.008203 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.008126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:56.070327 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.070299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:56.075418 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.075399    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tbxgw_2f3f1bb4-c75e-4b61-9db9-034c63028124/node-ca/0.log"
Aug 04 11:33:56.475734 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.475667    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7dznr_15f98335-5a43-4491-839c-11a71132cb4b/serve-healthcheck-canary/0.log"
Aug 04 11:33:56.672716 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.672685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:56.672900 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.672733    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:33:56.672900 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:56.672793    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:56.672900 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:56.672871    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 11:33:56.673023 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:56.672874    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:00.672855832 +0000 UTC m=+54.243768614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : secret "insights-runtime-extractor-tls" not found
Aug 04 11:33:56.673023 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:56.672949    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:00.672934242 +0000 UTC m=+54.243847023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : configmap "kube-rbac-proxy" not found
Aug 04 11:33:56.673271 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:56.673252    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:57.867995 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:57.867950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:33:58.184579 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:58.184579 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:33:58.184756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184582    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:33:58.184756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:33:58.184756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:33:58.184756 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184665    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.18464547 +0000 UTC m=+67.755558255 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : configmap references non-existent config key: service-ca.crt
Aug 04 11:33:58.184756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:33:58.184756 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184732    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184767    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184782    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls podName:4880a511-b249-4367-9811-55fcbc037524 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.184768564 +0000 UTC m=+67.755681349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-dsb2c" (UID: "4880a511-b249-4367-9811-55fcbc037524") : secret "cluster-monitoring-operator-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184731    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184823    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184808    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert podName:a9605480-7983-409a-b5da-e941c4f7b331 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.184796708 +0000 UTC m=+67.755709495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b2z74" (UID: "a9605480-7983-409a-b5da-e941c4f7b331") : secret "networking-console-plugin-cert" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:58.184802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184849    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls podName:2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.184837429 +0000 UTC m=+67.755750238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls") pod "dns-default-tgnj9" (UID: "2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba") : secret "dns-default-metrics-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184858    2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184851    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184892    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd88f78d6-4h9mv: secret "image-registry-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184898    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs podName:26e8828f-fa45-4972-8389-98db03ce35f3 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.184869874 +0000 UTC m=+67.755782671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs") pod "router-default-6bf9477994-mzx8r" (UID: "26e8828f-fa45-4972-8389-98db03ce35f3") : secret "router-metrics-certs-default" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184920    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls podName:55b006b1-9d28-428c-89ec-07c23c853b5b nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.18490852 +0000 UTC m=+67.755821309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-pbmmv" (UID: "55b006b1-9d28-428c-89ec-07c23c853b5b") : secret "samples-operator-tls" not found
Aug 04 11:33:58.185052 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:33:58.184939    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls podName:5946213e-2cba-468c-9683-fe335d33ba70 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.18493102 +0000 UTC m=+67.755843816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls") pod "image-registry-5fd88f78d6-4h9mv" (UID: "5946213e-2cba-468c-9683-fe335d33ba70") : secret "image-registry-tls" not found
Aug 04 11:33:59.590501 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:33:59.590469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:00.020765 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.020732    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:00.311696 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.311602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:00.371457 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.371412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:00.438586 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.438558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:00.540876 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.540839    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:00.707563 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.707484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:34:00.707563 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.707525    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:34:00.707946 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:34:00.707642    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 11:34:00.707946 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:34:00.707707    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls podName:d2c90932-c01e-4d72-ba89-ebce20441511 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:08.707690037 +0000 UTC m=+62.278602827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7k5jv" (UID: "d2c90932-c01e-4d72-ba89-ebce20441511") : secret "insights-runtime-extractor-tls" not found
Aug 04 11:34:00.708099 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.708080    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2c90932-c01e-4d72-ba89-ebce20441511-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:34:00.717792 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:00.717765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:01.516992 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:01.516955    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:01.920690 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:01.920597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:02.331253 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:02.331222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:02.522269 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:02.522241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:02.614471 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:02.614406    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:34:02.614471 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:02.614457    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:34:02.614824 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:02.614803    2558 scope.go:117] "RemoveContainer" containerID="931b46a3b83cc136fe2db23db63bd5beedc99735e978158820b7edbb6db6c3cd"
Aug 04 11:34:02.931071 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:02.930983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:03.058250 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.058219    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:03.284497 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.284469    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:34:03.284887 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.284873    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/1.log"
Aug 04 11:34:03.284943 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.284905    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a86e8b9-802f-442f-b6e8-4daf7b39d025" containerID="06d0f034fd09c4e192961d015370dd994f21875b1055c7409d455c2d2fbb9766" exitCode=255
Aug 04 11:34:03.284978 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.284960    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" event={"ID":"5a86e8b9-802f-442f-b6e8-4daf7b39d025","Type":"ContainerDied","Data":"06d0f034fd09c4e192961d015370dd994f21875b1055c7409d455c2d2fbb9766"}
Aug 04 11:34:03.285012 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.285004    2558 scope.go:117] "RemoveContainer" containerID="931b46a3b83cc136fe2db23db63bd5beedc99735e978158820b7edbb6db6c3cd"
Aug 04 11:34:03.285343 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.285322    2558 scope.go:117] "RemoveContainer" containerID="06d0f034fd09c4e192961d015370dd994f21875b1055c7409d455c2d2fbb9766"
Aug 04 11:34:03.285514 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:34:03.285495    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-rjvth_openshift-console-operator(5a86e8b9-802f-442f-b6e8-4daf7b39d025)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" podUID="5a86e8b9-802f-442f-b6e8-4daf7b39d025"
Aug 04 11:34:03.521327 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.521292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:03.583476 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.583413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:03.721510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.721485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:03.915848 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:03.915781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:04.289972 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:04.289947    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:34:04.318133 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:04.318105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:04.376296 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:04.376256    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:04.718301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:04.718207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:04.919246 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:04.919216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:05.118348 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:05.118317    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:05.218976 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:05.218946    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:05.319460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:05.319429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:06.225472 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:06.225441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:06.324227 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:06.324198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:06.517925 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:06.517898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:06.719569 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:06.719538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:06.918252 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:06.918194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:07.122774 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:07.122741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:08.120590 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.120558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:08.323817 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.323780    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:08.715453 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.715419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:08.785215 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.785184    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:34:08.787467 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.787445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2c90932-c01e-4d72-ba89-ebce20441511-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k5jv\" (UID: \"d2c90932-c01e-4d72-ba89-ebce20441511\") " pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:34:08.861050 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.861022    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fqqnm\""
Aug 04 11:34:08.868548 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.868530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7k5jv"
Aug 04 11:34:08.919324 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.919258    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:08.993059 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:08.993038    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7k5jv"]
Aug 04 11:34:09.168480 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:09.168454    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cbd22"
Aug 04 11:34:09.305912 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:09.305883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k5jv" event={"ID":"d2c90932-c01e-4d72-ba89-ebce20441511","Type":"ContainerStarted","Data":"de92ba5b627b562e519ef8970973ab4ffc12e744bd8d24d339d1787811a9d47d"}
Aug 04 11:34:09.306081 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:09.305935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k5jv" event={"ID":"d2c90932-c01e-4d72-ba89-ebce20441511","Type":"ContainerStarted","Data":"db7c6ce47c27a2ea78298714ce0939c04b53567796fdf9fbb5b9da51b880f9a1"}
Aug 04 11:34:09.318022 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:09.317995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:10.141587 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:10.141555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:10.311489 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:10.311451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k5jv" event={"ID":"d2c90932-c01e-4d72-ba89-ebce20441511","Type":"ContainerStarted","Data":"ea5badcade19b3c49d9a901ec789807f71e922e383fa197534025ca16343c0b6"}
Aug 04 11:34:10.316393 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:10.316357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:10.521167 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:10.521134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:10.717772 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:10.717741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:10.974204 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:10.974179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:11.074964 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:11.074933    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:11.316624 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:11.316586    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k5jv" event={"ID":"d2c90932-c01e-4d72-ba89-ebce20441511","Type":"ContainerStarted","Data":"058ffccba34e077080265330fd458656fcd8c3f3be9d610f2d09a0ebea9df10b"}
Aug 04 11:34:11.335601 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:11.335556    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7k5jv" podStartSLOduration=17.50864676 podStartE2EDuration="19.335540885s" totalImagesPullingTime="1.826894125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:11.333508777 +0000 UTC m=+64.904421580" watchObservedRunningTime="2026-08-04 11:34:11.335540885 +0000 UTC m=+64.906453689"
Aug 04 11:34:12.614366 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.614335    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:34:12.614744 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.614387    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:34:12.614744 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.614697    2558 scope.go:117] "RemoveContainer" containerID="06d0f034fd09c4e192961d015370dd994f21875b1055c7409d455c2d2fbb9766"
Aug 04 11:34:12.614868 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:34:12.614849    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-rjvth_openshift-console-operator(5a86e8b9-802f-442f-b6e8-4daf7b39d025)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" podUID="5a86e8b9-802f-442f-b6e8-4daf7b39d025"
Aug 04 11:34:12.617950 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.617926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:34:12.620371 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.620355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 11:34:12.630618 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.630598    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a-original-pull-secret\") pod \"global-pull-secret-syncer-j6nsk\" (UID: \"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a\") " pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:34:12.718657 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.718628    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:34:12.721510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.721492    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 11:34:12.731644 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.731624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9918b3-753f-4772-a082-1e922de1494d-metrics-certs\") pod \"network-metrics-daemon-s7lbd\" (UID: \"cd9918b3-753f-4772-a082-1e922de1494d\") " pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:34:12.799014 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.798989    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j6nsk"
Aug 04 11:34:12.807538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.807521    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9rm6g\""
Aug 04 11:34:12.815822 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.815790    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7lbd"
Aug 04 11:34:12.819186 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.819159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:34:12.821924 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.821898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sxm9\" (UniqueName: \"kubernetes.io/projected/b9c920ad-279e-4d13-a77c-96ddaadbe728-kube-api-access-2sxm9\") pod \"network-check-target-gb5hk\" (UID: \"b9c920ad-279e-4d13-a77c-96ddaadbe728\") " pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:34:12.925445 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.925340    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j6nsk"]
Aug 04 11:34:12.927972 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:12.927946    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6006cd_0e2f_4b6b_8771_aa0cbb86f77a.slice/crio-d362f6fe187a51fea798877b706f0540a5cfa3266e3a9c487627b730ac93849d WatchSource:0}: Error finding container d362f6fe187a51fea798877b706f0540a5cfa3266e3a9c487627b730ac93849d: Status 404 returned error can't find the container with id d362f6fe187a51fea798877b706f0540a5cfa3266e3a9c487627b730ac93849d
Aug 04 11:34:12.944882 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:12.944863    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7lbd"]
Aug 04 11:34:12.947207 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:12.947187    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9918b3_753f_4772_a082_1e922de1494d.slice/crio-5c00b58110950dfd863182ec56720d9dd708bf0be516261335a1fb5381e01deb WatchSource:0}: Error finding container 5c00b58110950dfd863182ec56720d9dd708bf0be516261335a1fb5381e01deb: Status 404 returned error can't find the container with id 5c00b58110950dfd863182ec56720d9dd708bf0be516261335a1fb5381e01deb
Aug 04 11:34:13.112336 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.112310    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zp2md\""
Aug 04 11:34:13.119551 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.119481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:34:13.239270 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.239195    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gb5hk"]
Aug 04 11:34:13.241693 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:13.241666    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c920ad_279e_4d13_a77c_96ddaadbe728.slice/crio-f1900331111d204082633505192d0ffc52dd4e396fa131e62b6b4f9344bcd29d WatchSource:0}: Error finding container f1900331111d204082633505192d0ffc52dd4e396fa131e62b6b4f9344bcd29d: Status 404 returned error can't find the container with id f1900331111d204082633505192d0ffc52dd4e396fa131e62b6b4f9344bcd29d
Aug 04 11:34:13.325616 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.325579    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gb5hk" event={"ID":"b9c920ad-279e-4d13-a77c-96ddaadbe728","Type":"ContainerStarted","Data":"e4a9c5a9bbe39bf1e8626d05464c1e6c5a54609a99521fadac87b6f1b47f2e4e"}
Aug 04 11:34:13.325789 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.325641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gb5hk" event={"ID":"b9c920ad-279e-4d13-a77c-96ddaadbe728","Type":"ContainerStarted","Data":"f1900331111d204082633505192d0ffc52dd4e396fa131e62b6b4f9344bcd29d"}
Aug 04 11:34:13.325789 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.325777    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:34:13.327039 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.327015    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7lbd" event={"ID":"cd9918b3-753f-4772-a082-1e922de1494d","Type":"ContainerStarted","Data":"5c00b58110950dfd863182ec56720d9dd708bf0be516261335a1fb5381e01deb"}
Aug 04 11:34:13.328170 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.328150    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j6nsk" event={"ID":"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a","Type":"ContainerStarted","Data":"d362f6fe187a51fea798877b706f0540a5cfa3266e3a9c487627b730ac93849d"}
Aug 04 11:34:13.342450 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:13.342407    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gb5hk" podStartSLOduration=66.342393705 podStartE2EDuration="1m6.342393705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:13.340658004 +0000 UTC m=+66.911570808" watchObservedRunningTime="2026-08-04 11:34:13.342393705 +0000 UTC m=+66.913306501"
Aug 04 11:34:14.232201 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:34:14.232634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232230    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:14.232634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:34:14.232634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:34:14.232634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:34:14.232634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:34:14.232634 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.232419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:14.233038 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.233005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e8828f-fa45-4972-8389-98db03ce35f3-service-ca-bundle\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:14.234927 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.234895    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9605480-7983-409a-b5da-e941c4f7b331-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b2z74\" (UID: \"a9605480-7983-409a-b5da-e941c4f7b331\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:34:14.235038 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.234959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"image-registry-5fd88f78d6-4h9mv\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") " pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:34:14.235117 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.235095    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba-metrics-tls\") pod \"dns-default-tgnj9\" (UID: \"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba\") " pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:34:14.235206 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.235192    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4880a511-b249-4367-9811-55fcbc037524-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-dsb2c\" (UID: \"4880a511-b249-4367-9811-55fcbc037524\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:34:14.235381 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.235363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26e8828f-fa45-4972-8389-98db03ce35f3-metrics-certs\") pod \"router-default-6bf9477994-mzx8r\" (UID: \"26e8828f-fa45-4972-8389-98db03ce35f3\") " pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:14.235416 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.235363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/55b006b1-9d28-428c-89ec-07c23c853b5b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-pbmmv\" (UID: \"55b006b1-9d28-428c-89ec-07c23c853b5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:34:14.369554 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.369527    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gql25\""
Aug 04 11:34:14.376784 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.376756    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:34:14.393660 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.393631    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ts7qd\""
Aug 04 11:34:14.401870 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.401846    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"
Aug 04 11:34:14.436561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.436252    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hll8p\""
Aug 04 11:34:14.444426 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.444401    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:34:14.445631 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.445579    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-btqpx\""
Aug 04 11:34:14.451527 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.451411    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zhbt4\""
Aug 04 11:34:14.451527 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.451447    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"
Aug 04 11:34:14.458258 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.458181    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:14.464813 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.464784    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kkwj7\""
Aug 04 11:34:14.473970 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.473945    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"
Aug 04 11:34:14.602421 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.601546    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5fd88f78d6-4h9mv"]
Aug 04 11:34:14.668997 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.667434    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c"]
Aug 04 11:34:14.669852 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.669552    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:14.670093 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:14.670066    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4880a511_b249_4367_9811_55fcbc037524.slice/crio-28ffef3ab439e0e3f848731a4767a7f5100ceb10386ef68120be8ea8bd9c65b5 WatchSource:0}: Error finding container 28ffef3ab439e0e3f848731a4767a7f5100ceb10386ef68120be8ea8bd9c65b5: Status 404 returned error can't find the container with id 28ffef3ab439e0e3f848731a4767a7f5100ceb10386ef68120be8ea8bd9c65b5
Aug 04 11:34:14.683067 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.683046    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tgnj9"]
Aug 04 11:34:14.759621 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.759543    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-b2z74"]
Aug 04 11:34:14.759621 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.759582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:14.768955 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:14.768922    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9605480_7983_409a_b5da_e941c4f7b331.slice/crio-d72f6538b4750627e05dd51dc778a09222df2293e2bc20dd3f83099916a604ca WatchSource:0}: Error finding container d72f6538b4750627e05dd51dc778a09222df2293e2bc20dd3f83099916a604ca: Status 404 returned error can't find the container with id d72f6538b4750627e05dd51dc778a09222df2293e2bc20dd3f83099916a604ca
Aug 04 11:34:14.820117 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.820075    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv"]
Aug 04 11:34:14.839216 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.839148    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6bf9477994-mzx8r"]
Aug 04 11:34:14.841101 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:14.841077    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e8828f_fa45_4972_8389_98db03ce35f3.slice/crio-19b906390346cc245bb75504278108a5eb48228aedc32ac762ec15d4763efc39 WatchSource:0}: Error finding container 19b906390346cc245bb75504278108a5eb48228aedc32ac762ec15d4763efc39: Status 404 returned error can't find the container with id 19b906390346cc245bb75504278108a5eb48228aedc32ac762ec15d4763efc39
Aug 04 11:34:14.902962 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:14.902910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:15.342091 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.342059    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6bf9477994-mzx8r" event={"ID":"26e8828f-fa45-4972-8389-98db03ce35f3","Type":"ContainerStarted","Data":"f3a3f963ce59bc9f53484de4ad0ecc3395dd26aaae107efa3b0fa89b2daf8de2"}
Aug 04 11:34:15.342565 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.342111    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6bf9477994-mzx8r" event={"ID":"26e8828f-fa45-4972-8389-98db03ce35f3","Type":"ContainerStarted","Data":"19b906390346cc245bb75504278108a5eb48228aedc32ac762ec15d4763efc39"}
Aug 04 11:34:15.345445 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.345411    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv" event={"ID":"55b006b1-9d28-428c-89ec-07c23c853b5b","Type":"ContainerStarted","Data":"9f1e0f71f6e8c70b8f0fa31da4ae5f57ee0000c26be283ecf2de0b93c8abae4a"}
Aug 04 11:34:15.347019 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.346960    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74" event={"ID":"a9605480-7983-409a-b5da-e941c4f7b331","Type":"ContainerStarted","Data":"d72f6538b4750627e05dd51dc778a09222df2293e2bc20dd3f83099916a604ca"}
Aug 04 11:34:15.349117 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.349051    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tgnj9" event={"ID":"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba","Type":"ContainerStarted","Data":"30cd0c1108db9e4c81ca13ea2c464910edcc5782bb33b135166c823a524230b4"}
Aug 04 11:34:15.351694 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.351633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c" event={"ID":"4880a511-b249-4367-9811-55fcbc037524","Type":"ContainerStarted","Data":"28ffef3ab439e0e3f848731a4767a7f5100ceb10386ef68120be8ea8bd9c65b5"}
Aug 04 11:34:15.354460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.354378    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7lbd" event={"ID":"cd9918b3-753f-4772-a082-1e922de1494d","Type":"ContainerStarted","Data":"114861929b9fe6bebd6638ea65cd22dd36d4a38044d762cef30ce00f9306d491"}
Aug 04 11:34:15.354460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.354419    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7lbd" event={"ID":"cd9918b3-753f-4772-a082-1e922de1494d","Type":"ContainerStarted","Data":"8563458523e2be52df96b62c5dffbc7ac8aed70ed95513e147598fca5d3419c1"}
Aug 04 11:34:15.358606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.358571    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" event={"ID":"5946213e-2cba-468c-9683-fe335d33ba70","Type":"ContainerStarted","Data":"8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca"}
Aug 04 11:34:15.358606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.358601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" event={"ID":"5946213e-2cba-468c-9683-fe335d33ba70","Type":"ContainerStarted","Data":"d9d9c7143520a78b6176bf5ba1a1a61a5e2b8cfdd3d4d1ce304425ed2c09a73c"}
Aug 04 11:34:15.358948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.358934    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:34:15.395445 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.395396    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6bf9477994-mzx8r" podStartSLOduration=48.395376114 podStartE2EDuration="48.395376114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:15.372254316 +0000 UTC m=+68.943167120" watchObservedRunningTime="2026-08-04 11:34:15.395376114 +0000 UTC m=+68.966288922"
Aug 04 11:34:15.396245 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.396204    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s7lbd" podStartSLOduration=67.080377116 podStartE2EDuration="1m8.396193114s" totalImagesPullingTime="1.315815998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:15.394605318 +0000 UTC m=+68.965518123" watchObservedRunningTime="2026-08-04 11:34:15.396193114 +0000 UTC m=+68.967105919"
Aug 04 11:34:15.420779 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.419324    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" podStartSLOduration=67.41930618 podStartE2EDuration="1m7.41930618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:15.41754074 +0000 UTC m=+68.988453548" watchObservedRunningTime="2026-08-04 11:34:15.41930618 +0000 UTC m=+68.990218986"
Aug 04 11:34:15.459152 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.459082    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:15.462438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.462414    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:15.521927 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.521894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:15.622714 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:15.622621    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:16.365299 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:16.365256    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:16.366449 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:16.366400    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6bf9477994-mzx8r"
Aug 04 11:34:16.593668 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:16.593636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:16.659151 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:16.659062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:19.380005 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.379967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j6nsk" event={"ID":"7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a","Type":"ContainerStarted","Data":"baa63daf289ff2e4502ee9eb9a552707e47ac8ae8dbeac3452cee3c57c12aedf"}
Aug 04 11:34:19.385045 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.384334    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv" event={"ID":"55b006b1-9d28-428c-89ec-07c23c853b5b","Type":"ContainerStarted","Data":"3017e3419431a6918b91b07f179cd3c70104aad75e5bf606f8fb1cc1f4398b8a"}
Aug 04 11:34:19.391010 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.390946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74" event={"ID":"a9605480-7983-409a-b5da-e941c4f7b331","Type":"ContainerStarted","Data":"8a0212270677b22b76881538f66f1d18a1d111a6dd1ca5caff9314635c2d00fb"}
Aug 04 11:34:19.396148 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.396071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tgnj9" event={"ID":"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba","Type":"ContainerStarted","Data":"612459a36d35c182a4083ecc46bf6af549ca6de8e6a5f2ee7a5fd7511a664f82"}
Aug 04 11:34:19.398249 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.397963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c" event={"ID":"4880a511-b249-4367-9811-55fcbc037524","Type":"ContainerStarted","Data":"f599f45078b0010c6692aabb4e3638008936a9b2d4d209eb83896426831f3ca9"}
Aug 04 11:34:19.401585 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.400994    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j6nsk" podStartSLOduration=66.139212001 podStartE2EDuration="1m12.400977813s" totalImagesPullingTime="6.261765812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:19.399469918 +0000 UTC m=+72.970382723" watchObservedRunningTime="2026-08-04 11:34:19.400977813 +0000 UTC m=+72.971890619"
Aug 04 11:34:19.440654 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.439977    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-dsb2c" podStartSLOduration=47.945260501 podStartE2EDuration="52.439955472s" totalImagesPullingTime="4.494694971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:19.421369624 +0000 UTC m=+72.992282431" watchObservedRunningTime="2026-08-04 11:34:19.439955472 +0000 UTC m=+73.010868277"
Aug 04 11:34:19.577366 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.577334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:19.662587 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.662497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:19.750895 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:19.750863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:20.143749 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.143717    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:20.403955 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.403854    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv" event={"ID":"55b006b1-9d28-428c-89ec-07c23c853b5b","Type":"ContainerStarted","Data":"ab050ef9c5dcb68bfcc96d1bf372abc5fb074ce268ce9d99883d6efb062f4795"}
Aug 04 11:34:20.405683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.405656    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tgnj9" event={"ID":"2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba","Type":"ContainerStarted","Data":"29a36a976520a885ccf11a0c008a304ca2d0076b53b37dce0f634a26272f6529"}
Aug 04 11:34:20.406035 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.406014    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:34:20.421549 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.421500    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b2z74" podStartSLOduration=42.028397599 podStartE2EDuration="46.421486496s" totalImagesPullingTime="4.393088897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:19.440741466 +0000 UTC m=+73.011654273" watchObservedRunningTime="2026-08-04 11:34:20.421486496 +0000 UTC m=+73.992399301"
Aug 04 11:34:20.421810 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.421781    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-pbmmv" podStartSLOduration=49.135785936 podStartE2EDuration="53.421772263s" totalImagesPullingTime="4.285986327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:20.420588688 +0000 UTC m=+73.991501491" watchObservedRunningTime="2026-08-04 11:34:20.421772263 +0000 UTC m=+73.992685113"
Aug 04 11:34:20.439954 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:20.439905    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tgnj9" podStartSLOduration=33.982271973 podStartE2EDuration="38.439891101s" totalImagesPullingTime="4.457619128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:20.438531266 +0000 UTC m=+74.009444072" watchObservedRunningTime="2026-08-04 11:34:20.439891101 +0000 UTC m=+74.010803904"
Aug 04 11:34:22.454891 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:22.454849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:22.514379 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:22.514353    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:24.789575 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:24.789542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:24.877523 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:24.877494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:24.938901 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:24.938858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:25.187792 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:25.187688    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:25.971931 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:25.971873    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:26.031925 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:26.031883    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:27.136645 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:27.136612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:27.718275 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:27.718215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:27.988727 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:27.988656    2558 scope.go:117] "RemoveContainer" containerID="06d0f034fd09c4e192961d015370dd994f21875b1055c7409d455c2d2fbb9766"
Aug 04 11:34:28.436555 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:28.436525    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:34:28.436940 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:28.436635    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" event={"ID":"5a86e8b9-802f-442f-b6e8-4daf7b39d025","Type":"ContainerStarted","Data":"781caebfc2f4b21b1b84014bcd13085533d7927cbca99330cf285e4a9c4825e1"}
Aug 04 11:34:28.437001 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:28.436962    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:34:28.456002 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:28.455940    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth" podStartSLOduration=53.91924575 podStartE2EDuration="1m1.455918694s" totalImagesPullingTime="7.536672944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:28.455343691 +0000 UTC m=+82.026256496" watchObservedRunningTime="2026-08-04 11:34:28.455918694 +0000 UTC m=+82.026831502"
Aug 04 11:34:29.014471 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.014447    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-rjvth"
Aug 04 11:34:29.180508 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.180472    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-rrkd5"]
Aug 04 11:34:29.188642 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.188609    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:29.191775 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.191746    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jckjv\""
Aug 04 11:34:29.191923 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.191817    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 04 11:34:29.191982 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.191915    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 04 11:34:29.193616 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.193555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:29.193982 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.193947    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-rrkd5"]
Aug 04 11:34:29.295357 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.295251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:29.351252 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.351213    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwg7p\" (UniqueName: \"kubernetes.io/projected/bc766a32-2e31-40b2-9ddd-d8294e7235b3-kube-api-access-vwg7p\") pod \"downloads-77c974ffcf-rrkd5\" (UID: \"bc766a32-2e31-40b2-9ddd-d8294e7235b3\") " pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:29.393117 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.393085    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:29.452419 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.452390    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwg7p\" (UniqueName: \"kubernetes.io/projected/bc766a32-2e31-40b2-9ddd-d8294e7235b3-kube-api-access-vwg7p\") pod \"downloads-77c974ffcf-rrkd5\" (UID: \"bc766a32-2e31-40b2-9ddd-d8294e7235b3\") " pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:29.462835 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.462810    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwg7p\" (UniqueName: \"kubernetes.io/projected/bc766a32-2e31-40b2-9ddd-d8294e7235b3-kube-api-access-vwg7p\") pod \"downloads-77c974ffcf-rrkd5\" (UID: \"bc766a32-2e31-40b2-9ddd-d8294e7235b3\") " pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:29.493988 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.493959    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:29.499929 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.499907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:29.621212 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.621188    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-rrkd5"]
Aug 04 11:34:29.623151 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:29.623123    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc766a32_2e31_40b2_9ddd_d8294e7235b3.slice/crio-97747123c13dfbf6db8f714c7d264f628a3104b5475d8bdb19e637a7cbd31bb7 WatchSource:0}: Error finding container 97747123c13dfbf6db8f714c7d264f628a3104b5475d8bdb19e637a7cbd31bb7: Status 404 returned error can't find the container with id 97747123c13dfbf6db8f714c7d264f628a3104b5475d8bdb19e637a7cbd31bb7
Aug 04 11:34:29.667768 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.667740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:29.769159 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.769118    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:29.892033 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:29.891928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.000100 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.000065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.168494 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.168413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.177367 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.177345    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"]
Aug 04 11:34:30.180432 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.180414    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.183403 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.183385    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Aug 04 11:34:30.184500 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.184485    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 11:34:30.184611 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.184518    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-824tk\""
Aug 04 11:34:30.184611 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.184540    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Aug 04 11:34:30.191612 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.191590    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"]
Aug 04 11:34:30.194017 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.194000    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"]
Aug 04 11:34:30.194108 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.194095    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.196739 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.196721    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2bxb7\""
Aug 04 11:34:30.197057 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.197033    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Aug 04 11:34:30.197160 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.197060    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Aug 04 11:34:30.197160 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.197122    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Aug 04 11:34:30.208229 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.208204    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"]
Aug 04 11:34:30.212088 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.212069    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9tr8b"]
Aug 04 11:34:30.214349 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.214333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.217089 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.217073    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 11:34:30.217180 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.217097    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 11:34:30.217367 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.217354    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 11:34:30.217422 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.217386    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tqxrb\""
Aug 04 11:34:30.265691 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.265664    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.332504 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.332426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.360948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.360915    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqwqr\" (UniqueName: \"kubernetes.io/projected/da96ae44-9d9e-4f18-84b4-ec7491343169-kube-api-access-rqwqr\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.360948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.360956    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.361198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361031    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-root\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361078    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.361198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a162196-18d7-4c5b-8bf5-21e87aa23d50-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.361198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-tls\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5qxf\" (UniqueName: \"kubernetes.io/projected/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-api-access-c5qxf\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361262    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a162196-18d7-4c5b-8bf5-21e87aa23d50-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshvm\" (UniqueName: \"kubernetes.io/projected/2a162196-18d7-4c5b-8bf5-21e87aa23d50-kube-api-access-sshvm\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-sys\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-wtmp\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-accelerators-collector-config\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da96ae44-9d9e-4f18-84b4-ec7491343169-metrics-client-ca\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361460    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a162196-18d7-4c5b-8bf5-21e87aa23d50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.361522 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361509    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-textfile\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.361940 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.361537    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.411489 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.411465    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tgnj9"
Aug 04 11:34:30.433558 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.433490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.448924 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.448720    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-rrkd5" event={"ID":"bc766a32-2e31-40b2-9ddd-d8294e7235b3","Type":"ContainerStarted","Data":"97747123c13dfbf6db8f714c7d264f628a3104b5475d8bdb19e637a7cbd31bb7"}
Aug 04 11:34:30.462433 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da96ae44-9d9e-4f18-84b4-ec7491343169-metrics-client-ca\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462451    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a162196-18d7-4c5b-8bf5-21e87aa23d50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462493    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-textfile\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqwqr\" (UniqueName: \"kubernetes.io/projected/da96ae44-9d9e-4f18-84b4-ec7491343169-kube-api-access-rqwqr\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462649    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-root\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462676    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a162196-18d7-4c5b-8bf5-21e87aa23d50-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-tls\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462798    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5qxf\" (UniqueName: \"kubernetes.io/projected/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-api-access-c5qxf\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.462840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462837    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a162196-18d7-4c5b-8bf5-21e87aa23d50-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sshvm\" (UniqueName: \"kubernetes.io/projected/2a162196-18d7-4c5b-8bf5-21e87aa23d50-kube-api-access-sshvm\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463014    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-sys\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-wtmp\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da96ae44-9d9e-4f18-84b4-ec7491343169-metrics-client-ca\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.463481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463076    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-accelerators-collector-config\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.463825 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463704    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.463825 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a162196-18d7-4c5b-8bf5-21e87aa23d50-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.463825 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463794    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-accelerators-collector-config\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.463975 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463915    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-textfile\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.464029 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.463980    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-sys\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.464029 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.462834    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-root\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.464181 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.464159    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-wtmp\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.464340 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.464265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.464656 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.464608    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.466714 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.466358    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a162196-18d7-4c5b-8bf5-21e87aa23d50-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.466714 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.466671    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-tls\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.467417 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.467339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da96ae44-9d9e-4f18-84b4-ec7491343169-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.467417 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.467334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.468076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.468027    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.469799 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.469772    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a162196-18d7-4c5b-8bf5-21e87aa23d50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.476116 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.476096    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5qxf\" (UniqueName: \"kubernetes.io/projected/77cf4aa6-6848-40fa-94b2-4a0907cddfd1-kube-api-access-c5qxf\") pod \"kube-state-metrics-6bd9664c4-bkbvv\" (UID: \"77cf4aa6-6848-40fa-94b2-4a0907cddfd1\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.476271 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.476249    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshvm\" (UniqueName: \"kubernetes.io/projected/2a162196-18d7-4c5b-8bf5-21e87aa23d50-kube-api-access-sshvm\") pod \"openshift-state-metrics-6bb4d7b4dd-kmgl7\" (UID: \"2a162196-18d7-4c5b-8bf5-21e87aa23d50\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.477147 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.477125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqwqr\" (UniqueName: \"kubernetes.io/projected/da96ae44-9d9e-4f18-84b4-ec7491343169-kube-api-access-rqwqr\") pod \"node-exporter-9tr8b\" (UID: \"da96ae44-9d9e-4f18-84b4-ec7491343169\") " pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.490105 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.490005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"
Aug 04 11:34:30.502482 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.502405    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"
Aug 04 11:34:30.523618 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.523584    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9tr8b"
Aug 04 11:34:30.533636 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.533609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.536691 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:30.536654    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda96ae44_9d9e_4f18_84b4_ec7491343169.slice/crio-f1580ea8fbfeac5d6b13680d52f05d0af8a2f5b17c3020edefeebaa82421a7fb WatchSource:0}: Error finding container f1580ea8fbfeac5d6b13680d52f05d0af8a2f5b17c3020edefeebaa82421a7fb: Status 404 returned error can't find the container with id f1580ea8fbfeac5d6b13680d52f05d0af8a2f5b17c3020edefeebaa82421a7fb
Aug 04 11:34:30.636888 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.636857    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:30.648444 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.648416    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7"]
Aug 04 11:34:30.653083 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:30.653054    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a162196_18d7_4c5b_8bf5_21e87aa23d50.slice/crio-4e7e0bc2d0542e006954c89e0cb802cb9facb0dcf792ff6c0a5136cc0bfc43a0 WatchSource:0}: Error finding container 4e7e0bc2d0542e006954c89e0cb802cb9facb0dcf792ff6c0a5136cc0bfc43a0: Status 404 returned error can't find the container with id 4e7e0bc2d0542e006954c89e0cb802cb9facb0dcf792ff6c0a5136cc0bfc43a0
Aug 04 11:34:30.677031 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.677005    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv"]
Aug 04 11:34:30.682552 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:30.682527    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77cf4aa6_6848_40fa_94b2_4a0907cddfd1.slice/crio-144f11bb9eb478be19d5dadfed54ea1ef91892b66fe6d8bd5e408566f29fad9a WatchSource:0}: Error finding container 144f11bb9eb478be19d5dadfed54ea1ef91892b66fe6d8bd5e408566f29fad9a: Status 404 returned error can't find the container with id 144f11bb9eb478be19d5dadfed54ea1ef91892b66fe6d8bd5e408566f29fad9a
Aug 04 11:34:30.959223 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:30.959143    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:31.076133 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.076099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:31.177490 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.177451    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:31.250948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.250865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:31.383508 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.383476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:31.456659 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.456622    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv" event={"ID":"77cf4aa6-6848-40fa-94b2-4a0907cddfd1","Type":"ContainerStarted","Data":"144f11bb9eb478be19d5dadfed54ea1ef91892b66fe6d8bd5e408566f29fad9a"}
Aug 04 11:34:31.460463 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.460433    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7" event={"ID":"2a162196-18d7-4c5b-8bf5-21e87aa23d50","Type":"ContainerStarted","Data":"4fc037ffc08f66115ec7593dc1eb4fbf197789306fb40bff77b4936c15e497ba"}
Aug 04 11:34:31.460603 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.460493    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7" event={"ID":"2a162196-18d7-4c5b-8bf5-21e87aa23d50","Type":"ContainerStarted","Data":"202203c66b3ee36bf19053858bf3020bf327b2523f58f9ecf445cc11ce1306e9"}
Aug 04 11:34:31.460603 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.460521    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7" event={"ID":"2a162196-18d7-4c5b-8bf5-21e87aa23d50","Type":"ContainerStarted","Data":"4e7e0bc2d0542e006954c89e0cb802cb9facb0dcf792ff6c0a5136cc0bfc43a0"}
Aug 04 11:34:31.462499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.462477    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9tr8b" event={"ID":"da96ae44-9d9e-4f18-84b4-ec7491343169","Type":"ContainerStarted","Data":"f1580ea8fbfeac5d6b13680d52f05d0af8a2f5b17c3020edefeebaa82421a7fb"}
Aug 04 11:34:31.483816 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:31.483299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:32.469669 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:32.469588    2558 generic.go:356] "Generic (PLEG): container finished" podID="da96ae44-9d9e-4f18-84b4-ec7491343169" containerID="515528450ecad9a7f50998182fbc3942d69a101b2f6a80ece04511f16749318f" exitCode=0
Aug 04 11:34:32.469669 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:32.469630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9tr8b" event={"ID":"da96ae44-9d9e-4f18-84b4-ec7491343169","Type":"ContainerDied","Data":"515528450ecad9a7f50998182fbc3942d69a101b2f6a80ece04511f16749318f"}
Aug 04 11:34:32.513967 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:32.513933    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:32.572096 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:32.572039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:33.202778 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.202746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:33.306714 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.306683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:33.478270 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.478189    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7" event={"ID":"2a162196-18d7-4c5b-8bf5-21e87aa23d50","Type":"ContainerStarted","Data":"45f98508e57e9f760b5e4c590baad01164aa27ad4d9603edb84bac7697091bfe"}
Aug 04 11:34:33.480564 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.480531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9tr8b" event={"ID":"da96ae44-9d9e-4f18-84b4-ec7491343169","Type":"ContainerStarted","Data":"e5c4517f85ced4c29cc852d8f656ce025ca190547d387ccaad91e0e60d459556"}
Aug 04 11:34:33.480724 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.480586    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9tr8b" event={"ID":"da96ae44-9d9e-4f18-84b4-ec7491343169","Type":"ContainerStarted","Data":"10be9b93626e7a511b4879824f66eb23de0f6a2bec4b32d08810792e300b9ca6"}
Aug 04 11:34:33.482969 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.482942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv" event={"ID":"77cf4aa6-6848-40fa-94b2-4a0907cddfd1","Type":"ContainerStarted","Data":"8346ffbe7d157a1b4ea8cd3247901310c7819a681d87117ca615228475742719"}
Aug 04 11:34:33.483094 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.482998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv" event={"ID":"77cf4aa6-6848-40fa-94b2-4a0907cddfd1","Type":"ContainerStarted","Data":"103abd1e3eda121e75f75d8eca584980b86559a42b4e473bb0541495998e998e"}
Aug 04 11:34:33.483094 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.483033    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv" event={"ID":"77cf4aa6-6848-40fa-94b2-4a0907cddfd1","Type":"ContainerStarted","Data":"f3229247140960f7f4498e340c87528563c4f84815ae8d60888d1faf98ee02d7"}
Aug 04 11:34:33.499005 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.498958    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-kmgl7" podStartSLOduration=1.803741782 podStartE2EDuration="3.498940138s" totalImagesPullingTime="1.695198356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:33.496822867 +0000 UTC m=+87.067735673" watchObservedRunningTime="2026-08-04 11:34:33.498940138 +0000 UTC m=+87.069852944"
Aug 04 11:34:33.501453 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.501417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:33.516661 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.516610    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-bkbvv" podStartSLOduration=1.712204075 podStartE2EDuration="3.51659338s" totalImagesPullingTime="1.804389305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:33.515262287 +0000 UTC m=+87.086175094" watchObservedRunningTime="2026-08-04 11:34:33.51659338 +0000 UTC m=+87.087506185"
Aug 04 11:34:33.540016 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.538328    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9tr8b" podStartSLOduration=2.301014923 podStartE2EDuration="3.538306672s" totalImagesPullingTime="1.153326344s" totalInitContainerRuntime="83.965405ms" isStatefulPod=true podCreationTimestamp="2026-08-04 11:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:33.53616993 +0000 UTC m=+87.107082735" watchObservedRunningTime="2026-08-04 11:34:33.538306672 +0000 UTC m=+87.109219476"
Aug 04 11:34:33.732399 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:33.732305    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:34.383001 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:34.382963    2558 patch_prober.go:28] interesting pod/image-registry-5fd88f78d6-4h9mv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 04 11:34:34.383169 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:34.383023    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" podUID="5946213e-2cba-468c-9683-fe335d33ba70" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 11:34:34.953943 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:34.953915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:35.053956 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:35.053919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:35.195438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:35.195402    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:36.441398 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:36.441362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:36.544932 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:36.544902    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5fd88f78d6-4h9mv"]
Aug 04 11:34:36.550413 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:36.550389    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:34:36.640822 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:36.640778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:36.739848 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:36.739767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:36.839754 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:36.839728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:37.311157 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:37.311122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:37.374196 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:37.374155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:39.550654 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.550614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:39.612601 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.612564    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:39.696335 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.696275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:39.791661 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.791620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:39.798751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.798722    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56b7d485f4-tjlqm"]
Aug 04 11:34:39.801430 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.801367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.805727 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.805700    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 04 11:34:39.805851 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.805797    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cj6dc\""
Aug 04 11:34:39.805851 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.805805    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 04 11:34:39.805851 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.805826    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 04 11:34:39.806109 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.806086    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 04 11:34:39.806227 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.806169    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 04 11:34:39.811777 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.811757    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b7d485f4-tjlqm"]
Aug 04 11:34:39.848861 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.848829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-oauth-config\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.849034 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.848888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-config\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.849034 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.848944    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-serving-cert\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.849034 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.849016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-service-ca\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.849195 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.849041    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxzt\" (UniqueName: \"kubernetes.io/projected/cd8b6069-82b8-415e-b192-1ea4448b8cd4-kube-api-access-kbxzt\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.849195 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.849070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-oauth-serving-cert\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.950204 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-service-ca\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.950398 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxzt\" (UniqueName: \"kubernetes.io/projected/cd8b6069-82b8-415e-b192-1ea4448b8cd4-kube-api-access-kbxzt\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.950398 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-oauth-serving-cert\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.950513 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-oauth-config\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.950513 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-config\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.950513 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-serving-cert\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.951008 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.950978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-service-ca\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.951244 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.951223    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-config\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.951561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.951537    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-oauth-serving-cert\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.953561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.953538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-oauth-config\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.953650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.953566    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-serving-cert\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:39.959240 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:39.959219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbxzt\" (UniqueName: \"kubernetes.io/projected/cd8b6069-82b8-415e-b192-1ea4448b8cd4-kube-api-access-kbxzt\") pod \"console-56b7d485f4-tjlqm\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") " pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:40.113755 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:40.113670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:34:40.979363 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:40.979330    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:41.036679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:41.036640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:41.908085 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:41.908049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:41.961556 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:41.961522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:42.645747 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:42.645715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:42.745499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:42.745463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:43.901980 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:43.901950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:43.954537 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:43.954500    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:44.337868 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:44.337842    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gb5hk"
Aug 04 11:34:46.164149 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.164124    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:46.264438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.264415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:46.279872 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.279847    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b7d485f4-tjlqm"]
Aug 04 11:34:46.281621 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:34:46.281595    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8b6069_82b8_415e_b192_1ea4448b8cd4.slice/crio-7d7de9de1b95c0a7122d17727c1cedfa89f561a266a5fa181fb2faafa7b45047 WatchSource:0}: Error finding container 7d7de9de1b95c0a7122d17727c1cedfa89f561a266a5fa181fb2faafa7b45047: Status 404 returned error can't find the container with id 7d7de9de1b95c0a7122d17727c1cedfa89f561a266a5fa181fb2faafa7b45047
Aug 04 11:34:46.543346 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.543310    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-rrkd5" event={"ID":"bc766a32-2e31-40b2-9ddd-d8294e7235b3","Type":"ContainerStarted","Data":"9e5ac4579c48a62e62044667830feed56cf29dc241947a9bf74ab9a448b63c75"}
Aug 04 11:34:46.543540 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.543493    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:46.544834 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.544804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b7d485f4-tjlqm" event={"ID":"cd8b6069-82b8-415e-b192-1ea4448b8cd4","Type":"ContainerStarted","Data":"7d7de9de1b95c0a7122d17727c1cedfa89f561a266a5fa181fb2faafa7b45047"}
Aug 04 11:34:46.562957 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.562901    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-rrkd5" podStartSLOduration=0.933304477 podStartE2EDuration="17.562886812s" totalImagesPullingTime="16.629582335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:46.562066368 +0000 UTC m=+100.132979173" watchObservedRunningTime="2026-08-04 11:34:46.562886812 +0000 UTC m=+100.133799616"
Aug 04 11:34:46.563898 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:46.563874    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-rrkd5"
Aug 04 11:34:48.321856 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:48.321778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:48.565371 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:48.565333    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:48.666345 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:48.666240    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:50.567887 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:50.567852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b7d485f4-tjlqm" event={"ID":"cd8b6069-82b8-415e-b192-1ea4448b8cd4","Type":"ContainerStarted","Data":"136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7"}
Aug 04 11:34:50.587544 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:50.587469    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56b7d485f4-tjlqm" podStartSLOduration=7.939988254 podStartE2EDuration="11.587448862s" totalImagesPullingTime="3.647460608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:50.586998298 +0000 UTC m=+104.157911105" watchObservedRunningTime="2026-08-04 11:34:50.587448862 +0000 UTC m=+104.158361667"
Aug 04 11:34:51.722295 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:51.722240    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:51.781510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:51.781471    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:52.891780 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:52.891741    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b7d485f4-tjlqm"]
Aug 04 11:34:55.973517 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:55.973483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:56.031349 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:56.031322    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:57.603552 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:57.603476    2558 generic.go:356] "Generic (PLEG): container finished" podID="238cdeaf-02dc-45e2-94d5-0fb95fe656a9" containerID="1690902e6b2ca052edd874d0a352395627d4554ace60763a423cdff695b13425" exitCode=0
Aug 04 11:34:57.603946 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:57.603548    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9" event={"ID":"238cdeaf-02dc-45e2-94d5-0fb95fe656a9","Type":"ContainerDied","Data":"1690902e6b2ca052edd874d0a352395627d4554ace60763a423cdff695b13425"}
Aug 04 11:34:57.603946 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:57.603867    2558 scope.go:117] "RemoveContainer" containerID="1690902e6b2ca052edd874d0a352395627d4554ace60763a423cdff695b13425"
Aug 04 11:34:58.493662 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:58.493636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:58.547054 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:58.547024    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:34:58.609102 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:58.609071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8xpt9" event={"ID":"238cdeaf-02dc-45e2-94d5-0fb95fe656a9","Type":"ContainerStarted","Data":"f36ca0a82450072546fec1acd115d916c31a8b2a8bc26c1bbfb02719b52665ae"}
Aug 04 11:34:59.709454 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:34:59.709398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:00.114635 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:00.114599    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:35:01.566606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.566563    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" podUID="5946213e-2cba-468c-9683-fe335d33ba70" containerName="registry" containerID="cri-o://8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca" gracePeriod=30
Aug 04 11:35:01.833593 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.833517    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:35:01.952156 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952119    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5946213e-2cba-468c-9683-fe335d33ba70-ca-trust-extracted\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952377 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952181    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-trusted-ca\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952377 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952214    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq67r\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-kube-api-access-fq67r\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952377 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952243    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-bound-sa-token\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952377 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-registry-certificates\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952377 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952324    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-image-registry-private-configuration\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952377 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952368    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952670 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952449    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-installation-pull-secrets\") pod \"5946213e-2cba-468c-9683-fe335d33ba70\" (UID: \"5946213e-2cba-468c-9683-fe335d33ba70\") "
Aug 04 11:35:01.952809 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952780    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:35:01.952878 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.952843    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:35:01.955032 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.954965    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:35:01.955032 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.954982    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-kube-api-access-fq67r" (OuterVolumeSpecName: "kube-api-access-fq67r") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "kube-api-access-fq67r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:35:01.955211 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.955059    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:35:01.955211 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.955079    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:35:01.955434 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.955416    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:35:01.960627 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:01.960605    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5946213e-2cba-468c-9683-fe335d33ba70-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5946213e-2cba-468c-9683-fe335d33ba70" (UID: "5946213e-2cba-468c-9683-fe335d33ba70"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:35:02.053802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053767    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-registry-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.053802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053796    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-installation-pull-secrets\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.053802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053807    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5946213e-2cba-468c-9683-fe335d33ba70-ca-trust-extracted\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.053802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053816    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-trusted-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.054081 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053823    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq67r\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-kube-api-access-fq67r\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.054081 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053842    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5946213e-2cba-468c-9683-fe335d33ba70-bound-sa-token\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.054081 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053850    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5946213e-2cba-468c-9683-fe335d33ba70-registry-certificates\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.054081 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.053859    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5946213e-2cba-468c-9683-fe335d33ba70-image-registry-private-configuration\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:02.626231 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.626200    2558 generic.go:356] "Generic (PLEG): container finished" podID="5946213e-2cba-468c-9683-fe335d33ba70" containerID="8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca" exitCode=0
Aug 04 11:35:02.626682 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.626242    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv"
Aug 04 11:35:02.626682 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.626288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" event={"ID":"5946213e-2cba-468c-9683-fe335d33ba70","Type":"ContainerDied","Data":"8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca"}
Aug 04 11:35:02.626682 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.626343    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5fd88f78d6-4h9mv" event={"ID":"5946213e-2cba-468c-9683-fe335d33ba70","Type":"ContainerDied","Data":"d9d9c7143520a78b6176bf5ba1a1a61a5e2b8cfdd3d4d1ce304425ed2c09a73c"}
Aug 04 11:35:02.626682 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.626363    2558 scope.go:117] "RemoveContainer" containerID="8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca"
Aug 04 11:35:02.635164 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.635144    2558 scope.go:117] "RemoveContainer" containerID="8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca"
Aug 04 11:35:02.635445 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:35:02.635426    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca\": container with ID starting with 8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca not found: ID does not exist" containerID="8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca"
Aug 04 11:35:02.635504 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.635453    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca"} err="failed to get container status \"8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca\": rpc error: code = NotFound desc = could not find container \"8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca\": container with ID starting with 8af2d2ef3c66f29ee08217070e8d31c241590fffcd72a8f9c6ac56f2ef0ac7ca not found: ID does not exist"
Aug 04 11:35:02.648694 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.648666    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5fd88f78d6-4h9mv"]
Aug 04 11:35:02.653194 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.653170    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5fd88f78d6-4h9mv"]
Aug 04 11:35:02.993138 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:02.993061    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5946213e-2cba-468c-9683-fe335d33ba70" path="/var/lib/kubelet/pods/5946213e-2cba-468c-9683-fe335d33ba70/volumes"
Aug 04 11:35:10.977131 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:10.977093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:11.039470 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:11.039438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:12.663545 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:12.663521    2558 generic.go:356] "Generic (PLEG): container finished" podID="e45dee60-cff9-46f2-820a-3f7e343524bd" containerID="fa246a5436c3f314e4b8506fe4c1c839b213b8792d134d221cab3616f46f7326" exitCode=0
Aug 04 11:35:12.663855 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:12.663590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp" event={"ID":"e45dee60-cff9-46f2-820a-3f7e343524bd","Type":"ContainerDied","Data":"fa246a5436c3f314e4b8506fe4c1c839b213b8792d134d221cab3616f46f7326"}
Aug 04 11:35:12.663932 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:12.663916    2558 scope.go:117] "RemoveContainer" containerID="fa246a5436c3f314e4b8506fe4c1c839b213b8792d134d221cab3616f46f7326"
Aug 04 11:35:13.672917 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:13.672878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-5jqsp" event={"ID":"e45dee60-cff9-46f2-820a-3f7e343524bd","Type":"ContainerStarted","Data":"acba1300ad5917129270523bc26c0a76c4a1a6b723faece4ff79b342aa61c0b4"}
Aug 04 11:35:17.914147 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:17.914102    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56b7d485f4-tjlqm" podUID="cd8b6069-82b8-415e-b192-1ea4448b8cd4" containerName="console" containerID="cri-o://136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7" gracePeriod=15
Aug 04 11:35:18.177399 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.177333    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:35:18.287038 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.286993    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-config\") pod \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") "
Aug 04 11:35:18.287038 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287045    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-service-ca\") pod \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") "
Aug 04 11:35:18.287336 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287083    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-serving-cert\") pod \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") "
Aug 04 11:35:18.287336 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287219    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-oauth-serving-cert\") pod \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") "
Aug 04 11:35:18.287336 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287254    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-oauth-config\") pod \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") "
Aug 04 11:35:18.287336 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287319    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbxzt\" (UniqueName: \"kubernetes.io/projected/cd8b6069-82b8-415e-b192-1ea4448b8cd4-kube-api-access-kbxzt\") pod \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\" (UID: \"cd8b6069-82b8-415e-b192-1ea4448b8cd4\") "
Aug 04 11:35:18.287548 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287461    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-config" (OuterVolumeSpecName: "console-config") pod "cd8b6069-82b8-415e-b192-1ea4448b8cd4" (UID: "cd8b6069-82b8-415e-b192-1ea4448b8cd4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:35:18.287606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287567    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cd8b6069-82b8-415e-b192-1ea4448b8cd4" (UID: "cd8b6069-82b8-415e-b192-1ea4448b8cd4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:35:18.287606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287585    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-config\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:18.287846 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.287815    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-service-ca" (OuterVolumeSpecName: "service-ca") pod "cd8b6069-82b8-415e-b192-1ea4448b8cd4" (UID: "cd8b6069-82b8-415e-b192-1ea4448b8cd4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:35:18.289473 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.289446    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8b6069-82b8-415e-b192-1ea4448b8cd4-kube-api-access-kbxzt" (OuterVolumeSpecName: "kube-api-access-kbxzt") pod "cd8b6069-82b8-415e-b192-1ea4448b8cd4" (UID: "cd8b6069-82b8-415e-b192-1ea4448b8cd4"). InnerVolumeSpecName "kube-api-access-kbxzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:35:18.289473 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.289465    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cd8b6069-82b8-415e-b192-1ea4448b8cd4" (UID: "cd8b6069-82b8-415e-b192-1ea4448b8cd4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:35:18.289473 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.289448    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cd8b6069-82b8-415e-b192-1ea4448b8cd4" (UID: "cd8b6069-82b8-415e-b192-1ea4448b8cd4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:35:18.388904 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.388869    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:18.388904 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.388899    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-serving-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:18.389126 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.388921    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd8b6069-82b8-415e-b192-1ea4448b8cd4-oauth-serving-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:18.389126 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.388944    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd8b6069-82b8-415e-b192-1ea4448b8cd4-console-oauth-config\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:18.389126 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.388952    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbxzt\" (UniqueName: \"kubernetes.io/projected/cd8b6069-82b8-415e-b192-1ea4448b8cd4-kube-api-access-kbxzt\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:35:18.580124 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.580092    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:18.681732 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.681699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:18.695965 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.695924    2558 generic.go:356] "Generic (PLEG): container finished" podID="cd8b6069-82b8-415e-b192-1ea4448b8cd4" containerID="136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7" exitCode=0
Aug 04 11:35:18.696141 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.696002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b7d485f4-tjlqm" event={"ID":"cd8b6069-82b8-415e-b192-1ea4448b8cd4","Type":"ContainerDied","Data":"136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7"}
Aug 04 11:35:18.696141 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.696082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b7d485f4-tjlqm" event={"ID":"cd8b6069-82b8-415e-b192-1ea4448b8cd4","Type":"ContainerDied","Data":"7d7de9de1b95c0a7122d17727c1cedfa89f561a266a5fa181fb2faafa7b45047"}
Aug 04 11:35:18.696141 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.696121    2558 scope.go:117] "RemoveContainer" containerID="136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7"
Aug 04 11:35:18.696359 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.696342    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b7d485f4-tjlqm"
Aug 04 11:35:18.705263 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.705244    2558 scope.go:117] "RemoveContainer" containerID="136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7"
Aug 04 11:35:18.705544 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:35:18.705523    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7\": container with ID starting with 136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7 not found: ID does not exist" containerID="136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7"
Aug 04 11:35:18.705594 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.705552    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7"} err="failed to get container status \"136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7\": rpc error: code = NotFound desc = could not find container \"136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7\": container with ID starting with 136fffe50ab23689f19ed3b50d358c7750c5096efe4c8a290d61a0ed09c6b5d7 not found: ID does not exist"
Aug 04 11:35:18.718653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.718633    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b7d485f4-tjlqm"]
Aug 04 11:35:18.724655 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.724635    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56b7d485f4-tjlqm"]
Aug 04 11:35:18.741468 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.741426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:18.993494 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:18.993415    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd8b6069-82b8-415e-b192-1ea4448b8cd4" path="/var/lib/kubelet/pods/cd8b6069-82b8-415e-b192-1ea4448b8cd4/volumes"
Aug 04 11:35:22.714832 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:22.714798    2558 generic.go:356] "Generic (PLEG): container finished" podID="33a40371-2939-448c-a467-d338dad327d1" containerID="82ac8cfdbe0f092c75bf0cd9cb9d9e5dddff7a6257bd49b6c11aeb072e2c710c" exitCode=0
Aug 04 11:35:22.715216 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:22.714855    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-klg2f" event={"ID":"33a40371-2939-448c-a467-d338dad327d1","Type":"ContainerDied","Data":"82ac8cfdbe0f092c75bf0cd9cb9d9e5dddff7a6257bd49b6c11aeb072e2c710c"}
Aug 04 11:35:22.715260 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:22.715212    2558 scope.go:117] "RemoveContainer" containerID="82ac8cfdbe0f092c75bf0cd9cb9d9e5dddff7a6257bd49b6c11aeb072e2c710c"
Aug 04 11:35:23.720727 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:23.720695    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-klg2f" event={"ID":"33a40371-2939-448c-a467-d338dad327d1","Type":"ContainerStarted","Data":"16f422198ec5b940ba614705e23592cd34f267ad2ad2e2772c3780c9a310739c"}
Aug 04 11:35:49.432257 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:49.432211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:49.911262 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:49.911230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:58.173569 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173531    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-796d4449d4-6fhlc"]
Aug 04 11:35:58.174069 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173826    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5946213e-2cba-468c-9683-fe335d33ba70" containerName="registry"
Aug 04 11:35:58.174069 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173837    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5946213e-2cba-468c-9683-fe335d33ba70" containerName="registry"
Aug 04 11:35:58.174069 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173859    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd8b6069-82b8-415e-b192-1ea4448b8cd4" containerName="console"
Aug 04 11:35:58.174069 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173866    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd8b6069-82b8-415e-b192-1ea4448b8cd4" containerName="console"
Aug 04 11:35:58.174069 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173910    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5946213e-2cba-468c-9683-fe335d33ba70" containerName="registry"
Aug 04 11:35:58.174069 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.173917    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd8b6069-82b8-415e-b192-1ea4448b8cd4" containerName="console"
Aug 04 11:35:58.175876 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.175856    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.179659 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.179635    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 04 11:35:58.179892 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.179874    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 04 11:35:58.185994 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.184897    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 04 11:35:58.185994 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.185423    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 04 11:35:58.185994 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.185628    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 04 11:35:58.185994 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.185814    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cj6dc\""
Aug 04 11:35:58.191234 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.191214    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 04 11:35:58.192464 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.192444    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796d4449d4-6fhlc"]
Aug 04 11:35:58.277981 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.277949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-trusted-ca-bundle\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.277981 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.277981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-oauth-serving-cert\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.278179 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.278005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-service-ca\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.278179 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.278023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-config\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.278179 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.278105    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-oauth-config\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.278179 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.278138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9xpn\" (UniqueName: \"kubernetes.io/projected/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-kube-api-access-n9xpn\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.278179 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.278157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-serving-cert\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.293160 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.293120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:58.378510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-oauth-config\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.378510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378511    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9xpn\" (UniqueName: \"kubernetes.io/projected/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-kube-api-access-n9xpn\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.378722 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378528    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-serving-cert\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.378722 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-trusted-ca-bundle\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.378722 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-oauth-serving-cert\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.378722 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378612    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-service-ca\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.378722 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.378635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-config\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.379485 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.379460    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-oauth-serving-cert\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.379594 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.379499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-config\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.379594 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.379514    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-service-ca\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.379594 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.379583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-trusted-ca-bundle\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.381011 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.380994    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-serving-cert\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.381108 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.381091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-oauth-config\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.387027 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.387004    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9xpn\" (UniqueName: \"kubernetes.io/projected/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-kube-api-access-n9xpn\") pod \"console-796d4449d4-6fhlc\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") " pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.391803 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.391784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:58.449291 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.449201    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:35:58.492324 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.492271    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:35:58.627888 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.625204    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796d4449d4-6fhlc"]
Aug 04 11:35:58.629713 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:35:58.629684    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb628438e_b2d7_423a_be41_e9a6e3cc7c9b.slice/crio-c73375ad96c104976c39d2dc8dd130d2e2962e00979d954262cc682de0736f03 WatchSource:0}: Error finding container c73375ad96c104976c39d2dc8dd130d2e2962e00979d954262cc682de0736f03: Status 404 returned error can't find the container with id c73375ad96c104976c39d2dc8dd130d2e2962e00979d954262cc682de0736f03
Aug 04 11:35:58.854439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.854400    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796d4449d4-6fhlc" event={"ID":"b628438e-b2d7-423a-be41-e9a6e3cc7c9b","Type":"ContainerStarted","Data":"5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63"}
Aug 04 11:35:58.854617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.854462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796d4449d4-6fhlc" event={"ID":"b628438e-b2d7-423a-be41-e9a6e3cc7c9b","Type":"ContainerStarted","Data":"c73375ad96c104976c39d2dc8dd130d2e2962e00979d954262cc682de0736f03"}
Aug 04 11:35:58.874195 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.874151    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-796d4449d4-6fhlc" podStartSLOduration=0.874135793 podStartE2EDuration="874.135793ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:35:58.87318427 +0000 UTC m=+172.444097075" watchObservedRunningTime="2026-08-04 11:35:58.874135793 +0000 UTC m=+172.445048597"
Aug 04 11:35:58.970550 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:35:58.970518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:06.516718 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:06.516685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:08.492964 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:08.492931    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:36:08.493355 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:08.493009    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:36:08.497911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:08.497885    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:36:08.897136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:08.897109    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:36:08.994925 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:08.994896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:09.098410 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:09.098364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:09.198715 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:09.198651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:12.090415 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:12.090380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:53.428796 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:53.428762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:36:53.488644 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:36:53.488611    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:04.648629 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:04.648596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:04.750168 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:04.750132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:12.865263 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.865231    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58d94f8766-rmq69"]
Aug 04 11:37:12.868102 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.868080    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.881079 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.881054    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d94f8766-rmq69"]
Aug 04 11:37:12.894253 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-serving-cert\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.894253 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8tr\" (UniqueName: \"kubernetes.io/projected/66445014-a469-4362-88cb-6ea74e8de74f-kube-api-access-bw8tr\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.894497 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-console-config\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.894497 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-trusted-ca-bundle\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.894497 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-oauth-config\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.894497 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894466    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-oauth-serving-cert\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.894730 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.894501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-service-ca\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.916859 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.916817    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:12.975809 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.975786    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:12.995471 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-serving-cert\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.995572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8tr\" (UniqueName: \"kubernetes.io/projected/66445014-a469-4362-88cb-6ea74e8de74f-kube-api-access-bw8tr\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.995572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-console-config\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.995572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995527    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-trusted-ca-bundle\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.995572 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-oauth-config\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.995756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-oauth-serving-cert\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.995756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.995615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-service-ca\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.996268 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.996246    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-console-config\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.996427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.996402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-oauth-serving-cert\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.996483 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.996464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-service-ca\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.996555 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.996539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-trusted-ca-bundle\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.998019 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.997998    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-serving-cert\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:12.998098 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:12.998013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-oauth-config\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:13.004414 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:13.004396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8tr\" (UniqueName: \"kubernetes.io/projected/66445014-a469-4362-88cb-6ea74e8de74f-kube-api-access-bw8tr\") pod \"console-58d94f8766-rmq69\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") " pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:13.181482 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:13.181405    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:13.305806 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:13.305776    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d94f8766-rmq69"]
Aug 04 11:37:13.308735 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:37:13.308696    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66445014_a469_4362_88cb_6ea74e8de74f.slice/crio-104a541896857a5a7d25c020bffc8cadbe89ddc2b443f4f22d71694ef648b787 WatchSource:0}: Error finding container 104a541896857a5a7d25c020bffc8cadbe89ddc2b443f4f22d71694ef648b787: Status 404 returned error can't find the container with id 104a541896857a5a7d25c020bffc8cadbe89ddc2b443f4f22d71694ef648b787
Aug 04 11:37:14.131004 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:14.130966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d94f8766-rmq69" event={"ID":"66445014-a469-4362-88cb-6ea74e8de74f","Type":"ContainerStarted","Data":"3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780"}
Aug 04 11:37:14.131451 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:14.131035    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d94f8766-rmq69" event={"ID":"66445014-a469-4362-88cb-6ea74e8de74f","Type":"ContainerStarted","Data":"104a541896857a5a7d25c020bffc8cadbe89ddc2b443f4f22d71694ef648b787"}
Aug 04 11:37:14.166770 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:14.166720    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58d94f8766-rmq69" podStartSLOduration=2.166704693 podStartE2EDuration="2.166704693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:37:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:37:14.164699993 +0000 UTC m=+247.735612799" watchObservedRunningTime="2026-08-04 11:37:14.166704693 +0000 UTC m=+247.737617497"
Aug 04 11:37:14.918840 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:14.918806    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:16.711668 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:16.711636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:16.784482 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:16.784454    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:23.182500 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:23.182453    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:23.182955 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:23.182521    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:23.187568 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:23.187547    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:24.170570 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:24.170542    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:37:24.226255 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:24.226218    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796d4449d4-6fhlc"]
Aug 04 11:37:24.342000 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:24.341951    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:24.442969 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:24.442884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:24.523260 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:24.523217    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:25.998425 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:25.998381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:26.098084 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:26.098051    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:47.169677 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:47.169645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:47.283212 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:47.283182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:37:49.247208 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.247140    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-796d4449d4-6fhlc" podUID="b628438e-b2d7-423a-be41-e9a6e3cc7c9b" containerName="console" containerID="cri-o://5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63" gracePeriod=15
Aug 04 11:37:49.483956 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.483929    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:37:49.602310 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602261    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-oauth-serving-cert\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602464 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602316    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-oauth-config\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602464 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602338    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-service-ca\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602455    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9xpn\" (UniqueName: \"kubernetes.io/projected/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-kube-api-access-n9xpn\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602546    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-config\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602666 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602581    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-serving-cert\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602666 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602615    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-trusted-ca-bundle\") pod \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\" (UID: \"b628438e-b2d7-423a-be41-e9a6e3cc7c9b\") "
Aug 04 11:37:49.602666 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602648    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-service-ca" (OuterVolumeSpecName: "service-ca") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:37:49.602829 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602703    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:37:49.602885 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602865    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:49.602936 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.602904    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-config" (OuterVolumeSpecName: "console-config") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:37:49.603091 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.603066    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:37:49.604586 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.604561    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:37:49.604683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.604605    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:37:49.604683 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.604632    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-kube-api-access-n9xpn" (OuterVolumeSpecName: "kube-api-access-n9xpn") pod "b628438e-b2d7-423a-be41-e9a6e3cc7c9b" (UID: "b628438e-b2d7-423a-be41-e9a6e3cc7c9b"). InnerVolumeSpecName "kube-api-access-n9xpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:37:49.703751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.703708    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-config\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:49.703751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.703746    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-serving-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:49.703751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.703756    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-trusted-ca-bundle\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:49.703751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.703764    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-oauth-serving-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:49.703988 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.703772    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-console-oauth-config\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:49.703988 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:49.703781    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9xpn\" (UniqueName: \"kubernetes.io/projected/b628438e-b2d7-423a-be41-e9a6e3cc7c9b-kube-api-access-n9xpn\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:37:50.263483 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.263449    2558 generic.go:356] "Generic (PLEG): container finished" podID="b628438e-b2d7-423a-be41-e9a6e3cc7c9b" containerID="5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63" exitCode=0
Aug 04 11:37:50.263483 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.263485    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796d4449d4-6fhlc" event={"ID":"b628438e-b2d7-423a-be41-e9a6e3cc7c9b","Type":"ContainerDied","Data":"5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63"}
Aug 04 11:37:50.263921 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.263495    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796d4449d4-6fhlc"
Aug 04 11:37:50.263921 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.263524    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796d4449d4-6fhlc" event={"ID":"b628438e-b2d7-423a-be41-e9a6e3cc7c9b","Type":"ContainerDied","Data":"c73375ad96c104976c39d2dc8dd130d2e2962e00979d954262cc682de0736f03"}
Aug 04 11:37:50.263921 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.263543    2558 scope.go:117] "RemoveContainer" containerID="5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63"
Aug 04 11:37:50.271967 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.271950    2558 scope.go:117] "RemoveContainer" containerID="5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63"
Aug 04 11:37:50.272191 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:37:50.272177    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63\": container with ID starting with 5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63 not found: ID does not exist" containerID="5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63"
Aug 04 11:37:50.272229 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.272198    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63"} err="failed to get container status \"5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63\": rpc error: code = NotFound desc = could not find container \"5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63\": container with ID starting with 5f978f6c8ea011b4e2ea41d7b22c41fcff8eff8f5b77de866ab6a2434be08b63 not found: ID does not exist"
Aug 04 11:37:50.285741 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.285717    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796d4449d4-6fhlc"]
Aug 04 11:37:50.289273 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.289252    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-796d4449d4-6fhlc"]
Aug 04 11:37:50.993173 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:37:50.993142    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b628438e-b2d7-423a-be41-e9a6e3cc7c9b" path="/var/lib/kubelet/pods/b628438e-b2d7-423a-be41-e9a6e3cc7c9b/volumes"
Aug 04 11:38:06.850196 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:06.850168    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:38:06.854746 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:06.854723    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:38:06.859086 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:06.859067    2558 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 11:38:09.100885 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:09.100848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:09.966716 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:09.966675    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:11.277322 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.277290    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"]
Aug 04 11:38:11.277854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.277710    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b628438e-b2d7-423a-be41-e9a6e3cc7c9b" containerName="console"
Aug 04 11:38:11.277854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.277728    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b628438e-b2d7-423a-be41-e9a6e3cc7c9b" containerName="console"
Aug 04 11:38:11.277854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.277823    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b628438e-b2d7-423a-be41-e9a6e3cc7c9b" containerName="console"
Aug 04 11:38:11.279609 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.279587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.282697 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.282678    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:38:11.283019 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.283003    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Aug 04 11:38:11.288355 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.288337    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2rdfm\""
Aug 04 11:38:11.304272 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.304247    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"]
Aug 04 11:38:11.368498 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.368465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5jt\" (UniqueName: \"kubernetes.io/projected/13814f7c-5611-4944-8126-6ae1f81b6ef4-kube-api-access-8x5jt\") pod \"cert-manager-operator-controller-manager-55c444bf86-7jmcq\" (UID: \"13814f7c-5611-4944-8126-6ae1f81b6ef4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.368679 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.368539    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13814f7c-5611-4944-8126-6ae1f81b6ef4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7jmcq\" (UID: \"13814f7c-5611-4944-8126-6ae1f81b6ef4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.469330 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.469291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13814f7c-5611-4944-8126-6ae1f81b6ef4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7jmcq\" (UID: \"13814f7c-5611-4944-8126-6ae1f81b6ef4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.469513 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.469351    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5jt\" (UniqueName: \"kubernetes.io/projected/13814f7c-5611-4944-8126-6ae1f81b6ef4-kube-api-access-8x5jt\") pod \"cert-manager-operator-controller-manager-55c444bf86-7jmcq\" (UID: \"13814f7c-5611-4944-8126-6ae1f81b6ef4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.469625 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.469605    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13814f7c-5611-4944-8126-6ae1f81b6ef4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7jmcq\" (UID: \"13814f7c-5611-4944-8126-6ae1f81b6ef4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.491514 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.491492    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5jt\" (UniqueName: \"kubernetes.io/projected/13814f7c-5611-4944-8126-6ae1f81b6ef4-kube-api-access-8x5jt\") pod \"cert-manager-operator-controller-manager-55c444bf86-7jmcq\" (UID: \"13814f7c-5611-4944-8126-6ae1f81b6ef4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.588675 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.588588    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"
Aug 04 11:38:11.739268 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.739245    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq"]
Aug 04 11:38:11.741952 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:38:11.741925    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13814f7c_5611_4944_8126_6ae1f81b6ef4.slice/crio-317350f9d42d5e4cf5a9282bc99c6172cadb21116f8c7d6341d6f836c2b26c39 WatchSource:0}: Error finding container 317350f9d42d5e4cf5a9282bc99c6172cadb21116f8c7d6341d6f836c2b26c39: Status 404 returned error can't find the container with id 317350f9d42d5e4cf5a9282bc99c6172cadb21116f8c7d6341d6f836c2b26c39
Aug 04 11:38:11.745437 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:11.745419    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 11:38:12.349193 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:12.349156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq" event={"ID":"13814f7c-5611-4944-8126-6ae1f81b6ef4","Type":"ContainerStarted","Data":"317350f9d42d5e4cf5a9282bc99c6172cadb21116f8c7d6341d6f836c2b26c39"}
Aug 04 11:38:13.136488 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:13.136459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:13.236155 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:13.236116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:15.364104 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:15.364069    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq" event={"ID":"13814f7c-5611-4944-8126-6ae1f81b6ef4","Type":"ContainerStarted","Data":"77d710ee282dd2cfef842bd83c9b598c84c3cb86e960712373ec04380418609d"}
Aug 04 11:38:15.393788 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:15.393688    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7jmcq" podStartSLOduration=0.995270365 podStartE2EDuration="4.393673637s" totalImagesPullingTime="3.398403272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:38:15.392421887 +0000 UTC m=+308.963334694" watchObservedRunningTime="2026-08-04 11:38:15.393673637 +0000 UTC m=+308.964586440"
Aug 04 11:38:16.207372 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.207335    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:16.212061 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.212038    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"]
Aug 04 11:38:16.214896 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.214881    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.219221 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.219204    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 04 11:38:16.220008 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.219993    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 04 11:38:16.220077 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.219993    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9vxtl\""
Aug 04 11:38:16.235210 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.235189    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"]
Aug 04 11:38:16.310302 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.310257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.310446 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.310358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.310446 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.310401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxjb9\" (UniqueName: \"kubernetes.io/projected/60246042-a50f-42c8-8de4-e055419a3b25-kube-api-access-kxjb9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.351260 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.351223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:16.411416 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.411384    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.411794 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.411445    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.411794 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.411488    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxjb9\" (UniqueName: \"kubernetes.io/projected/60246042-a50f-42c8-8de4-e055419a3b25-kube-api-access-kxjb9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.411866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.411837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.411866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.411855    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.425547 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.425521    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxjb9\" (UniqueName: \"kubernetes.io/projected/60246042-a50f-42c8-8de4-e055419a3b25-kube-api-access-kxjb9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.523494 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.523463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:16.673256 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:16.673217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"]
Aug 04 11:38:16.675918 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:38:16.675876    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60246042_a50f_42c8_8de4_e055419a3b25.slice/crio-e5169eaafdb7600bc1ef160bea7167ea3dcaee45aedf10c7718ca658b0bceec2 WatchSource:0}: Error finding container e5169eaafdb7600bc1ef160bea7167ea3dcaee45aedf10c7718ca658b0bceec2: Status 404 returned error can't find the container with id e5169eaafdb7600bc1ef160bea7167ea3dcaee45aedf10c7718ca658b0bceec2
Aug 04 11:38:17.373997 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:17.373955    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p" event={"ID":"60246042-a50f-42c8-8de4-e055419a3b25","Type":"ContainerStarted","Data":"e5169eaafdb7600bc1ef160bea7167ea3dcaee45aedf10c7718ca658b0bceec2"}
Aug 04 11:38:18.229876 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.229836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:18.421979 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.421945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:18.892056 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.892022    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"]
Aug 04 11:38:18.895032 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.895007    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:18.898299 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.898227    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 04 11:38:18.898627 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.898572    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 04 11:38:18.899692 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.899672    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hx6jx\""
Aug 04 11:38:18.906650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:18.906629    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"]
Aug 04 11:38:19.031550 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.031520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrwg\" (UniqueName: \"kubernetes.io/projected/ee297b8f-9977-45b8-acf2-42bf9a4a3d97-kube-api-access-2xrwg\") pod \"cert-manager-cainjector-559cdd8c87-jrbc2\" (UID: \"ee297b8f-9977-45b8-acf2-42bf9a4a3d97\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.031717 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.031591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee297b8f-9977-45b8-acf2-42bf9a4a3d97-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jrbc2\" (UID: \"ee297b8f-9977-45b8-acf2-42bf9a4a3d97\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.132872 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.132839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee297b8f-9977-45b8-acf2-42bf9a4a3d97-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jrbc2\" (UID: \"ee297b8f-9977-45b8-acf2-42bf9a4a3d97\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.133023 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.132886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrwg\" (UniqueName: \"kubernetes.io/projected/ee297b8f-9977-45b8-acf2-42bf9a4a3d97-kube-api-access-2xrwg\") pod \"cert-manager-cainjector-559cdd8c87-jrbc2\" (UID: \"ee297b8f-9977-45b8-acf2-42bf9a4a3d97\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.143936 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.143874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee297b8f-9977-45b8-acf2-42bf9a4a3d97-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jrbc2\" (UID: \"ee297b8f-9977-45b8-acf2-42bf9a4a3d97\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.143936 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.143925    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrwg\" (UniqueName: \"kubernetes.io/projected/ee297b8f-9977-45b8-acf2-42bf9a4a3d97-kube-api-access-2xrwg\") pod \"cert-manager-cainjector-559cdd8c87-jrbc2\" (UID: \"ee297b8f-9977-45b8-acf2-42bf9a4a3d97\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.211588 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.211553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"
Aug 04 11:38:19.360728 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.360702    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2"]
Aug 04 11:38:19.362839 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:38:19.362811    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee297b8f_9977_45b8_acf2_42bf9a4a3d97.slice/crio-19704a709806f9be70f3c12e4ac46e173129daa39906820bca29e001e1e5be95 WatchSource:0}: Error finding container 19704a709806f9be70f3c12e4ac46e173129daa39906820bca29e001e1e5be95: Status 404 returned error can't find the container with id 19704a709806f9be70f3c12e4ac46e173129daa39906820bca29e001e1e5be95
Aug 04 11:38:19.384205 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.384177    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2" event={"ID":"ee297b8f-9977-45b8-acf2-42bf9a4a3d97","Type":"ContainerStarted","Data":"19704a709806f9be70f3c12e4ac46e173129daa39906820bca29e001e1e5be95"}
Aug 04 11:38:19.500046 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.499957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:19.566719 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:19.566682    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:20.235389 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.235348    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sjd87"]
Aug 04 11:38:20.238100 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.238005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.241245 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.241221    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6h24r\""
Aug 04 11:38:20.248002 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.247865    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sjd87"]
Aug 04 11:38:20.345016 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.344971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwz4w\" (UniqueName: \"kubernetes.io/projected/896ac8fa-5713-4ae3-ade6-bfad27fba662-kube-api-access-qwz4w\") pod \"cert-manager-webhook-6486ffd997-sjd87\" (UID: \"896ac8fa-5713-4ae3-ade6-bfad27fba662\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.345198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.345072    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/896ac8fa-5713-4ae3-ade6-bfad27fba662-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sjd87\" (UID: \"896ac8fa-5713-4ae3-ade6-bfad27fba662\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.446600 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.446542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/896ac8fa-5713-4ae3-ade6-bfad27fba662-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sjd87\" (UID: \"896ac8fa-5713-4ae3-ade6-bfad27fba662\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.447134 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.446658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwz4w\" (UniqueName: \"kubernetes.io/projected/896ac8fa-5713-4ae3-ade6-bfad27fba662-kube-api-access-qwz4w\") pod \"cert-manager-webhook-6486ffd997-sjd87\" (UID: \"896ac8fa-5713-4ae3-ade6-bfad27fba662\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.456210 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.456151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/896ac8fa-5713-4ae3-ade6-bfad27fba662-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sjd87\" (UID: \"896ac8fa-5713-4ae3-ade6-bfad27fba662\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.456413 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.456386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwz4w\" (UniqueName: \"kubernetes.io/projected/896ac8fa-5713-4ae3-ade6-bfad27fba662-kube-api-access-qwz4w\") pod \"cert-manager-webhook-6486ffd997-sjd87\" (UID: \"896ac8fa-5713-4ae3-ade6-bfad27fba662\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.551827 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.551723    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:20.709024 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:20.708987    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sjd87"]
Aug 04 11:38:20.713177 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:38:20.713145    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod896ac8fa_5713_4ae3_ade6_bfad27fba662.slice/crio-125993157d32126d964c6e7c4ebbadce1beda3be80831e67996f1718482e95f7 WatchSource:0}: Error finding container 125993157d32126d964c6e7c4ebbadce1beda3be80831e67996f1718482e95f7: Status 404 returned error can't find the container with id 125993157d32126d964c6e7c4ebbadce1beda3be80831e67996f1718482e95f7
Aug 04 11:38:21.396554 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:21.396491    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87" event={"ID":"896ac8fa-5713-4ae3-ade6-bfad27fba662","Type":"ContainerStarted","Data":"125993157d32126d964c6e7c4ebbadce1beda3be80831e67996f1718482e95f7"}
Aug 04 11:38:23.396707 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:23.396673    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:23.646347 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:23.646309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:24.413322 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.413289    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2" event={"ID":"ee297b8f-9977-45b8-acf2-42bf9a4a3d97","Type":"ContainerStarted","Data":"7162666288dc78744b3373c4c99c0b49ee7f16c05c37b5e6b089ef4ca670ef61"}
Aug 04 11:38:24.415099 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.414986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87" event={"ID":"896ac8fa-5713-4ae3-ade6-bfad27fba662","Type":"ContainerStarted","Data":"0c12be8eef66a7e3fc20217155c563d683392d84d9475c7f0c3754b68866a1d7"}
Aug 04 11:38:24.415099 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.415035    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:24.416698 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.416674    2558 generic.go:356] "Generic (PLEG): container finished" podID="60246042-a50f-42c8-8de4-e055419a3b25" containerID="ff1562f955248d00446d2f47594ca6efba06aebc35d5f1dacf422bc0f045bfb2" exitCode=0
Aug 04 11:38:24.416822 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.416704    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p" event={"ID":"60246042-a50f-42c8-8de4-e055419a3b25","Type":"ContainerDied","Data":"ff1562f955248d00446d2f47594ca6efba06aebc35d5f1dacf422bc0f045bfb2"}
Aug 04 11:38:24.438639 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.438589    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jrbc2" podStartSLOduration=1.589305336 podStartE2EDuration="6.438575046s" totalImagesPullingTime="4.84926971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:38:24.43744422 +0000 UTC m=+318.008357024" watchObservedRunningTime="2026-08-04 11:38:24.438575046 +0000 UTC m=+318.009487850"
Aug 04 11:38:24.493038 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:24.492933    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87" podStartSLOduration=0.984938056 podStartE2EDuration="4.49291298s" totalImagesPullingTime="3.507974924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:38:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:38:24.465213422 +0000 UTC m=+318.036126281" watchObservedRunningTime="2026-08-04 11:38:24.49291298 +0000 UTC m=+318.063825785"
Aug 04 11:38:25.174159 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:25.174131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:25.325529 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:25.325497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:25.824753 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:25.824719    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:27.431401 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:27.431369    2558 generic.go:356] "Generic (PLEG): container finished" podID="60246042-a50f-42c8-8de4-e055419a3b25" containerID="481215030c0a95ea7da04e23426c760f07be6f77800ecabfb56960f36892e3d9" exitCode=0
Aug 04 11:38:27.431401 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:27.431406    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p" event={"ID":"60246042-a50f-42c8-8de4-e055419a3b25","Type":"ContainerDied","Data":"481215030c0a95ea7da04e23426c760f07be6f77800ecabfb56960f36892e3d9"}
Aug 04 11:38:30.423602 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:30.423567    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sjd87"
Aug 04 11:38:34.467857 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:34.467779    2558 generic.go:356] "Generic (PLEG): container finished" podID="60246042-a50f-42c8-8de4-e055419a3b25" containerID="25bdcf1c0fc6effbe442e3f93c674e969330c215cba74458e3f9264bfbddc7d1" exitCode=0
Aug 04 11:38:34.468181 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:34.467863    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p" event={"ID":"60246042-a50f-42c8-8de4-e055419a3b25","Type":"ContainerDied","Data":"25bdcf1c0fc6effbe442e3f93c674e969330c215cba74458e3f9264bfbddc7d1"}
Aug 04 11:38:35.598961 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.598927    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:35.681168 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.681135    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-util\") pod \"60246042-a50f-42c8-8de4-e055419a3b25\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") "
Aug 04 11:38:35.681352 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.681176    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-bundle\") pod \"60246042-a50f-42c8-8de4-e055419a3b25\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") "
Aug 04 11:38:35.681352 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.681206    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxjb9\" (UniqueName: \"kubernetes.io/projected/60246042-a50f-42c8-8de4-e055419a3b25-kube-api-access-kxjb9\") pod \"60246042-a50f-42c8-8de4-e055419a3b25\" (UID: \"60246042-a50f-42c8-8de4-e055419a3b25\") "
Aug 04 11:38:35.681636 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.681603    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-bundle" (OuterVolumeSpecName: "bundle") pod "60246042-a50f-42c8-8de4-e055419a3b25" (UID: "60246042-a50f-42c8-8de4-e055419a3b25"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:38:35.683413 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.683389    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60246042-a50f-42c8-8de4-e055419a3b25-kube-api-access-kxjb9" (OuterVolumeSpecName: "kube-api-access-kxjb9") pod "60246042-a50f-42c8-8de4-e055419a3b25" (UID: "60246042-a50f-42c8-8de4-e055419a3b25"). InnerVolumeSpecName "kube-api-access-kxjb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:38:35.686551 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.686527    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-util" (OuterVolumeSpecName: "util") pod "60246042-a50f-42c8-8de4-e055419a3b25" (UID: "60246042-a50f-42c8-8de4-e055419a3b25"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:38:35.782226 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.782190    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-util\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:38:35.782226 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.782223    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60246042-a50f-42c8-8de4-e055419a3b25-bundle\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:38:35.782226 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:35.782233    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxjb9\" (UniqueName: \"kubernetes.io/projected/60246042-a50f-42c8-8de4-e055419a3b25-kube-api-access-kxjb9\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:38:36.477502 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:36.477462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p" event={"ID":"60246042-a50f-42c8-8de4-e055419a3b25","Type":"ContainerDied","Data":"e5169eaafdb7600bc1ef160bea7167ea3dcaee45aedf10c7718ca658b0bceec2"}
Aug 04 11:38:36.477502 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:36.477488    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb57p"
Aug 04 11:38:36.477502 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:36.477506    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5169eaafdb7600bc1ef160bea7167ea3dcaee45aedf10c7718ca658b0bceec2"
Aug 04 11:38:41.507361 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:41.507325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:41.607639 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:41.607607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:47.358294 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:47.358248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:47.455685 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:47.455632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:48.984106 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984049    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"]
Aug 04 11:38:48.984481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984391    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="extract"
Aug 04 11:38:48.984481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984404    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="extract"
Aug 04 11:38:48.984481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984420    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="util"
Aug 04 11:38:48.984481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984426    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="util"
Aug 04 11:38:48.984481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984437    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="pull"
Aug 04 11:38:48.984481 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984442    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="pull"
Aug 04 11:38:48.984668 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.984502    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="60246042-a50f-42c8-8de4-e055419a3b25" containerName="extract"
Aug 04 11:38:48.989357 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.989332    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:48.990591 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.990565    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:48.990696 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.990612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5kpk\" (UniqueName: \"kubernetes.io/projected/157f91f0-3225-44de-aebe-fd680d23cf8d-kube-api-access-d5kpk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:48.990696 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.990662    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:48.992204 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.992184    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 04 11:38:48.992204 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.992200    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9vxtl\""
Aug 04 11:38:48.993556 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.993535    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 04 11:38:48.996371 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:48.996351    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"]
Aug 04 11:38:49.091236 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.091205    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.091466 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.091253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5kpk\" (UniqueName: \"kubernetes.io/projected/157f91f0-3225-44de-aebe-fd680d23cf8d-kube-api-access-d5kpk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.091466 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.091311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.091695 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.091674    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.091695 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.091690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.107360 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.107336    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5kpk\" (UniqueName: \"kubernetes.io/projected/157f91f0-3225-44de-aebe-fd680d23cf8d-kube-api-access-d5kpk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.299911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.299871    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:49.428648 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.428620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"]
Aug 04 11:38:49.431795 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:38:49.431768    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157f91f0_3225_44de_aebe_fd680d23cf8d.slice/crio-2793a4ac626f5b43ddbbf510645e6a61a552c4af44cceddc8314c18879d61a78 WatchSource:0}: Error finding container 2793a4ac626f5b43ddbbf510645e6a61a552c4af44cceddc8314c18879d61a78: Status 404 returned error can't find the container with id 2793a4ac626f5b43ddbbf510645e6a61a552c4af44cceddc8314c18879d61a78
Aug 04 11:38:49.534718 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.534688    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52" event={"ID":"157f91f0-3225-44de-aebe-fd680d23cf8d","Type":"ContainerStarted","Data":"76b8f7e9a009eeb7d15c75cd4e05aa831a88a057a3613a3c77b3a792841c02fc"}
Aug 04 11:38:49.534827 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.534750    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52" event={"ID":"157f91f0-3225-44de-aebe-fd680d23cf8d","Type":"ContainerStarted","Data":"2793a4ac626f5b43ddbbf510645e6a61a552c4af44cceddc8314c18879d61a78"}
Aug 04 11:38:49.737714 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:49.737632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:38:50.540182 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:50.540147    2558 generic.go:356] "Generic (PLEG): container finished" podID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerID="76b8f7e9a009eeb7d15c75cd4e05aa831a88a057a3613a3c77b3a792841c02fc" exitCode=0
Aug 04 11:38:50.540736 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:50.540235    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52" event={"ID":"157f91f0-3225-44de-aebe-fd680d23cf8d","Type":"ContainerDied","Data":"76b8f7e9a009eeb7d15c75cd4e05aa831a88a057a3613a3c77b3a792841c02fc"}
Aug 04 11:38:53.554882 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:53.554843    2558 generic.go:356] "Generic (PLEG): container finished" podID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerID="da6e6a409965f07e29feed6ee5399ba084c496b2b0bf1dad3e78265c12d3802c" exitCode=0
Aug 04 11:38:53.555353 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:53.554932    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52" event={"ID":"157f91f0-3225-44de-aebe-fd680d23cf8d","Type":"ContainerDied","Data":"da6e6a409965f07e29feed6ee5399ba084c496b2b0bf1dad3e78265c12d3802c"}
Aug 04 11:38:54.561737 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:54.561696    2558 generic.go:356] "Generic (PLEG): container finished" podID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerID="edb8a9cbf4909443a49d0e6558860d3a535cb59ce3240f3767e8f4337ca15de7" exitCode=0
Aug 04 11:38:54.562112 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:54.561776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52" event={"ID":"157f91f0-3225-44de-aebe-fd680d23cf8d","Type":"ContainerDied","Data":"edb8a9cbf4909443a49d0e6558860d3a535cb59ce3240f3767e8f4337ca15de7"}
Aug 04 11:38:55.691047 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.691024    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:55.740373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.740347    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-bundle\") pod \"157f91f0-3225-44de-aebe-fd680d23cf8d\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") "
Aug 04 11:38:55.740505 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.740391    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-util\") pod \"157f91f0-3225-44de-aebe-fd680d23cf8d\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") "
Aug 04 11:38:55.740505 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.740423    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5kpk\" (UniqueName: \"kubernetes.io/projected/157f91f0-3225-44de-aebe-fd680d23cf8d-kube-api-access-d5kpk\") pod \"157f91f0-3225-44de-aebe-fd680d23cf8d\" (UID: \"157f91f0-3225-44de-aebe-fd680d23cf8d\") "
Aug 04 11:38:55.741092 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.741063    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-bundle" (OuterVolumeSpecName: "bundle") pod "157f91f0-3225-44de-aebe-fd680d23cf8d" (UID: "157f91f0-3225-44de-aebe-fd680d23cf8d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:38:55.742456 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.742430    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/157f91f0-3225-44de-aebe-fd680d23cf8d-kube-api-access-d5kpk" (OuterVolumeSpecName: "kube-api-access-d5kpk") pod "157f91f0-3225-44de-aebe-fd680d23cf8d" (UID: "157f91f0-3225-44de-aebe-fd680d23cf8d"). InnerVolumeSpecName "kube-api-access-d5kpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:38:55.747641 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.747601    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-util" (OuterVolumeSpecName: "util") pod "157f91f0-3225-44de-aebe-fd680d23cf8d" (UID: "157f91f0-3225-44de-aebe-fd680d23cf8d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:38:55.841802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.841714    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-bundle\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:38:55.841802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.841753    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/157f91f0-3225-44de-aebe-fd680d23cf8d-util\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:38:55.841802 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:55.841765    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5kpk\" (UniqueName: \"kubernetes.io/projected/157f91f0-3225-44de-aebe-fd680d23cf8d-kube-api-access-d5kpk\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:38:56.572356 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:56.572317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52" event={"ID":"157f91f0-3225-44de-aebe-fd680d23cf8d","Type":"ContainerDied","Data":"2793a4ac626f5b43ddbbf510645e6a61a552c4af44cceddc8314c18879d61a78"}
Aug 04 11:38:56.572356 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:56.572342    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9d52"
Aug 04 11:38:56.572356 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:38:56.572364    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2793a4ac626f5b43ddbbf510645e6a61a552c4af44cceddc8314c18879d61a78"
Aug 04 11:39:04.114126 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114033    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"]
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114366    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="pull"
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114377    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="pull"
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114386    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="util"
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114391    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="util"
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114403    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="extract"
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114409    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="extract"
Aug 04 11:39:04.114512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.114465    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="157f91f0-3225-44de-aebe-fd680d23cf8d" containerName="extract"
Aug 04 11:39:04.135892 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.135859    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.145042 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.145022    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 04 11:39:04.145173 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.145023    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 04 11:39:04.146717 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.146697    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 04 11:39:04.146717 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.146710    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 04 11:39:04.146834 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.146721    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tgvcd\""
Aug 04 11:39:04.148804 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.148789    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 04 11:39:04.165049 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.165025    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"]
Aug 04 11:39:04.206301 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.206247    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvwm7\" (UniqueName: \"kubernetes.io/projected/f8483b2d-b3b8-43c5-9e87-df05963993f7-kube-api-access-wvwm7\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.206438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.206340    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8483b2d-b3b8-43c5-9e87-df05963993f7-cert\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.206438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.206372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8483b2d-b3b8-43c5-9e87-df05963993f7-metrics-cert\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.206515 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.206440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f8483b2d-b3b8-43c5-9e87-df05963993f7-manager-config\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.307108 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.307074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f8483b2d-b3b8-43c5-9e87-df05963993f7-manager-config\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.307318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.307124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvwm7\" (UniqueName: \"kubernetes.io/projected/f8483b2d-b3b8-43c5-9e87-df05963993f7-kube-api-access-wvwm7\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.307318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.307167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8483b2d-b3b8-43c5-9e87-df05963993f7-cert\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.307318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.307186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8483b2d-b3b8-43c5-9e87-df05963993f7-metrics-cert\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.307885 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.307861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f8483b2d-b3b8-43c5-9e87-df05963993f7-manager-config\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.309699 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.309679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8483b2d-b3b8-43c5-9e87-df05963993f7-cert\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.309855 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.309833    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8483b2d-b3b8-43c5-9e87-df05963993f7-metrics-cert\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.324805 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.324780    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvwm7\" (UniqueName: \"kubernetes.io/projected/f8483b2d-b3b8-43c5-9e87-df05963993f7-kube-api-access-wvwm7\") pod \"lws-controller-manager-d5b557dbd-dthbm\" (UID: \"f8483b2d-b3b8-43c5-9e87-df05963993f7\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.444801 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.444727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:04.642187 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:39:04.642157    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8483b2d_b3b8_43c5_9e87_df05963993f7.slice/crio-975ec220c3b251451719d530e309abfcf3403e21e68fe7ca3bd17e5bec98e6f2 WatchSource:0}: Error finding container 975ec220c3b251451719d530e309abfcf3403e21e68fe7ca3bd17e5bec98e6f2: Status 404 returned error can't find the container with id 975ec220c3b251451719d530e309abfcf3403e21e68fe7ca3bd17e5bec98e6f2
Aug 04 11:39:04.663645 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.663620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"]
Aug 04 11:39:04.698118 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.698048    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:04.826110 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:04.826076    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:05.148792 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:05.148757    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:05.264519 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:05.264469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:05.610249 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:05.610219    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm" event={"ID":"f8483b2d-b3b8-43c5-9e87-df05963993f7","Type":"ContainerStarted","Data":"975ec220c3b251451719d530e309abfcf3403e21e68fe7ca3bd17e5bec98e6f2"}
Aug 04 11:39:05.764599 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:05.764527    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.073592 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.073561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.170853 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.170815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.232854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.232826    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"]
Aug 04 11:39:06.248531 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.248510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.251073 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.251048    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 11:39:06.252325 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.252293    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 11:39:06.255117 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.255095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"]
Aug 04 11:39:06.258636 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.258517    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 11:39:06.258636 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.258535    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zv97q\""
Aug 04 11:39:06.258636 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.258629    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 11:39:06.325107 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.325025    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.325107 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.325066    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-tmp\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.325107 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.325098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-manifests\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.325355 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.325174    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8v5n\" (UniqueName: \"kubernetes.io/projected/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-kube-api-access-f8v5n\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.325355 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.325209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.383673 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.383639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.426354 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.426325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-manifests\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.426511 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.426367    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8v5n\" (UniqueName: \"kubernetes.io/projected/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-kube-api-access-f8v5n\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.426511 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.426389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.426511 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.426419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.426511 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.426456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-tmp\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.426801 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.426778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-manifests\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.428789 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.428762    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-tmp\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.428919 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.428901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.428958 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.428922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.434560 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.434539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8v5n\" (UniqueName: \"kubernetes.io/projected/6c48cdbf-906c-4a05-a7fb-6c2354dfe639-kube-api-access-f8v5n\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-vqgwl\" (UID: \"6c48cdbf-906c-4a05-a7fb-6c2354dfe639\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.482740 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.482698    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.560756 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.560732    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:06.592777 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.592732    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.616235 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.616190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm" event={"ID":"f8483b2d-b3b8-43c5-9e87-df05963993f7","Type":"ContainerStarted","Data":"51f94df1287891c7ecc156fba8e276eaea3e33fc5d3d2197d8090b40011a1fbc"}
Aug 04 11:39:06.695319 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.695245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:06.732540 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:06.732511    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"]
Aug 04 11:39:06.736012 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:39:06.735987    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c48cdbf_906c_4a05_a7fb_6c2354dfe639.slice/crio-9df23f5790750c7bd36f1c54b29b6e9f226d54100908451a8a61d2bba2629040 WatchSource:0}: Error finding container 9df23f5790750c7bd36f1c54b29b6e9f226d54100908451a8a61d2bba2629040: Status 404 returned error can't find the container with id 9df23f5790750c7bd36f1c54b29b6e9f226d54100908451a8a61d2bba2629040
Aug 04 11:39:07.556806 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:07.556757    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:07.625094 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:07.625045    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl" event={"ID":"6c48cdbf-906c-4a05-a7fb-6c2354dfe639","Type":"ContainerStarted","Data":"9df23f5790750c7bd36f1c54b29b6e9f226d54100908451a8a61d2bba2629040"}
Aug 04 11:39:07.625306 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:07.625145    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:07.653327 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:07.653261    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm" podStartSLOduration=1.796654063 podStartE2EDuration="3.653242433s" totalImagesPullingTime="1.85658837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:39:07.651089404 +0000 UTC m=+361.222002224" watchObservedRunningTime="2026-08-04 11:39:07.653242433 +0000 UTC m=+361.224155238"
Aug 04 11:39:07.653770 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:07.653739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:09.636714 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:09.636678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl" event={"ID":"6c48cdbf-906c-4a05-a7fb-6c2354dfe639","Type":"ContainerStarted","Data":"5b90ef82f0834e9536a1ec8a564f8b10713a280587c48530b8c8c7c84399e465"}
Aug 04 11:39:10.643971 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:10.643935    2558 generic.go:356] "Generic (PLEG): container finished" podID="6c48cdbf-906c-4a05-a7fb-6c2354dfe639" containerID="5b90ef82f0834e9536a1ec8a564f8b10713a280587c48530b8c8c7c84399e465" exitCode=0
Aug 04 11:39:10.644365 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:10.644007    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl" event={"ID":"6c48cdbf-906c-4a05-a7fb-6c2354dfe639","Type":"ContainerDied","Data":"5b90ef82f0834e9536a1ec8a564f8b10713a280587c48530b8c8c7c84399e465"}
Aug 04 11:39:11.650553 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:11.650518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl" event={"ID":"6c48cdbf-906c-4a05-a7fb-6c2354dfe639","Type":"ContainerStarted","Data":"bd3e558cc1c0ca7aff820b66d25a1b76c105f655ef3847bc170b3a59eb146fd3"}
Aug 04 11:39:11.651035 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:11.650672    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:11.678613 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:11.678561    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl" podStartSLOduration=1.466864015 podStartE2EDuration="5.678545713s" totalImagesPullingTime="3.155248042s" totalInitContainerRuntime="1.056433656s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:39:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:39:11.676793282 +0000 UTC m=+365.247706110" watchObservedRunningTime="2026-08-04 11:39:11.678545713 +0000 UTC m=+365.249458517"
Aug 04 11:39:18.632930 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:18.632901    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-dthbm"
Aug 04 11:39:22.657614 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:22.657581    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-vqgwl"
Aug 04 11:39:25.376116 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:25.376083    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:25.551095 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:25.551053    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:36.362231 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.362199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:36.430068 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.430035    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"]
Aug 04 11:39:36.433521 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.433496    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.436370 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.436347    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 04 11:39:36.436501 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.436388    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 04 11:39:36.436501 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.436417    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t8btz\""
Aug 04 11:39:36.443066 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.443044    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"]
Aug 04 11:39:36.589826 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.589788    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m86rs\" (UniqueName: \"kubernetes.io/projected/af47002b-71c8-4f17-9c84-28897ba189aa-kube-api-access-m86rs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.589980 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.589896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af47002b-71c8-4f17-9c84-28897ba189aa-tls-certs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.590025 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.589977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af47002b-71c8-4f17-9c84-28897ba189aa-tmp\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.609156 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.609110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:36.691080 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.690996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af47002b-71c8-4f17-9c84-28897ba189aa-tmp\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.691080 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.691060    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m86rs\" (UniqueName: \"kubernetes.io/projected/af47002b-71c8-4f17-9c84-28897ba189aa-kube-api-access-m86rs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.691303 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.691089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af47002b-71c8-4f17-9c84-28897ba189aa-tls-certs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.691303 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:39:36.691187    2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Aug 04 11:39:36.691303 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:39:36.691248    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af47002b-71c8-4f17-9c84-28897ba189aa-tls-certs podName:af47002b-71c8-4f17-9c84-28897ba189aa nodeName:}" failed. No retries permitted until 2026-08-04 11:39:37.191232013 +0000 UTC m=+390.762144795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/af47002b-71c8-4f17-9c84-28897ba189aa-tls-certs") pod "kube-auth-proxy-96bfd8458-l6tqk" (UID: "af47002b-71c8-4f17-9c84-28897ba189aa") : secret "kube-auth-proxy-tls" not found
Aug 04 11:39:36.692565 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.692540    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:36.693370 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.693353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af47002b-71c8-4f17-9c84-28897ba189aa-tmp\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.705053 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.705033    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m86rs\" (UniqueName: \"kubernetes.io/projected/af47002b-71c8-4f17-9c84-28897ba189aa-kube-api-access-m86rs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:36.966193 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:36.966114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:37.064169 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.064133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:37.195925 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.195891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af47002b-71c8-4f17-9c84-28897ba189aa-tls-certs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:37.199022 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.198992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af47002b-71c8-4f17-9c84-28897ba189aa-tls-certs\") pod \"kube-auth-proxy-96bfd8458-l6tqk\" (UID: \"af47002b-71c8-4f17-9c84-28897ba189aa\") " pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:37.320048 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.320018    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:37.345034 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.345005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"
Aug 04 11:39:37.411078 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.411044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:37.488364 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.488339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk"]
Aug 04 11:39:37.490432 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:39:37.490406    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf47002b_71c8_4f17_9c84_28897ba189aa.slice/crio-926d8fabc0259275c2555daea8a5200b077ce3f50226edca07683b786993841f WatchSource:0}: Error finding container 926d8fabc0259275c2555daea8a5200b077ce3f50226edca07683b786993841f: Status 404 returned error can't find the container with id 926d8fabc0259275c2555daea8a5200b077ce3f50226edca07683b786993841f
Aug 04 11:39:37.769944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.769889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk" event={"ID":"af47002b-71c8-4f17-9c84-28897ba189aa","Type":"ContainerStarted","Data":"926d8fabc0259275c2555daea8a5200b077ce3f50226edca07683b786993841f"}
Aug 04 11:39:37.866609 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.866576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:37.953515 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:37.953480    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:39.102430 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:39.102339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:39.706564 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:39.706526    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:39.924119 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:39.924074    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:39.990827 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:39.990750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:40.788202 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:40.788172    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk" event={"ID":"af47002b-71c8-4f17-9c84-28897ba189aa","Type":"ContainerStarted","Data":"e2de7d98f257fa6a3c9ff44374360c44090bb6d7149a10483d2b443dd24088a0"}
Aug 04 11:39:40.807897 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:40.807851    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk" podStartSLOduration=1.999137044 podStartE2EDuration="4.80783288s" totalImagesPullingTime="2.808695836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:39:40.806531129 +0000 UTC m=+394.377443933" watchObservedRunningTime="2026-08-04 11:39:40.80783288 +0000 UTC m=+394.378745687"
Aug 04 11:39:45.194252 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:45.194220    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:45.330439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:45.330410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:46.256350 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:46.256322    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:46.451346 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:46.451244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:46.547366 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:46.547331    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:47.874997 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:47.874966    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:48.003612 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:48.003583    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:48.180098 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:48.180003    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:48.397110 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:48.397076    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:48.526742 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:48.526709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:51.056977 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:51.056943    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d94f8766-rmq69"]
Aug 04 11:39:51.198759 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:51.198728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:39:51.334438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:39:51.334365    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:16.077289 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.077228    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58d94f8766-rmq69" podUID="66445014-a469-4362-88cb-6ea74e8de74f" containerName="console" containerID="cri-o://3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780" gracePeriod=15
Aug 04 11:40:16.314832 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.314806    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:40:16.417820 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417731    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-oauth-serving-cert\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.417820 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417765    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-trusted-ca-bundle\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.417820 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417799    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw8tr\" (UniqueName: \"kubernetes.io/projected/66445014-a469-4362-88cb-6ea74e8de74f-kube-api-access-bw8tr\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.417820 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417819    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-oauth-config\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.418150 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417845    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-console-config\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.418150 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417871    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-service-ca\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.418150 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.417889    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-serving-cert\") pod \"66445014-a469-4362-88cb-6ea74e8de74f\" (UID: \"66445014-a469-4362-88cb-6ea74e8de74f\") "
Aug 04 11:40:16.418330 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.418164    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:40:16.418330 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.418305    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:40:16.418436 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.418409    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-service-ca" (OuterVolumeSpecName: "service-ca") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:40:16.418499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.418483    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-console-config" (OuterVolumeSpecName: "console-config") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 11:40:16.420120 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.420091    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:40:16.420432 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.420407    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:40:16.420432 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.420426    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66445014-a469-4362-88cb-6ea74e8de74f-kube-api-access-bw8tr" (OuterVolumeSpecName: "kube-api-access-bw8tr") pod "66445014-a469-4362-88cb-6ea74e8de74f" (UID: "66445014-a469-4362-88cb-6ea74e8de74f"). InnerVolumeSpecName "kube-api-access-bw8tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:40:16.518742 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518711    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw8tr\" (UniqueName: \"kubernetes.io/projected/66445014-a469-4362-88cb-6ea74e8de74f-kube-api-access-bw8tr\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.518742 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518737    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-oauth-config\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.518742 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518746    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-console-config\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.518742 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518755    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.519018 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518764    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66445014-a469-4362-88cb-6ea74e8de74f-console-serving-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.519018 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518772    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-oauth-serving-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.519018 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.518781    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66445014-a469-4362-88cb-6ea74e8de74f-trusted-ca-bundle\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:40:16.963675 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.963639    2558 generic.go:356] "Generic (PLEG): container finished" podID="66445014-a469-4362-88cb-6ea74e8de74f" containerID="3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780" exitCode=0
Aug 04 11:40:16.963675 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.963679    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d94f8766-rmq69"
Aug 04 11:40:16.963886 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.963684    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d94f8766-rmq69" event={"ID":"66445014-a469-4362-88cb-6ea74e8de74f","Type":"ContainerDied","Data":"3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780"}
Aug 04 11:40:16.963886 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.963731    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d94f8766-rmq69" event={"ID":"66445014-a469-4362-88cb-6ea74e8de74f","Type":"ContainerDied","Data":"104a541896857a5a7d25c020bffc8cadbe89ddc2b443f4f22d71694ef648b787"}
Aug 04 11:40:16.963886 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.963753    2558 scope.go:117] "RemoveContainer" containerID="3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780"
Aug 04 11:40:16.973093 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.973078    2558 scope.go:117] "RemoveContainer" containerID="3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780"
Aug 04 11:40:16.973347 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:40:16.973328    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780\": container with ID starting with 3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780 not found: ID does not exist" containerID="3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780"
Aug 04 11:40:16.973405 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.973356    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780"} err="failed to get container status \"3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780\": rpc error: code = NotFound desc = could not find container \"3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780\": container with ID starting with 3bdbcc20ae9b9392e371bf7a29b4ad38686c61fac7a626ec3a15ac39bb9df780 not found: ID does not exist"
Aug 04 11:40:16.985083 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.985061    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d94f8766-rmq69"]
Aug 04 11:40:16.993698 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:16.993679    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58d94f8766-rmq69"]
Aug 04 11:40:18.993420 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:18.993377    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66445014-a469-4362-88cb-6ea74e8de74f" path="/var/lib/kubelet/pods/66445014-a469-4362-88cb-6ea74e8de74f/volumes"
Aug 04 11:40:27.089967 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:27.089936    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:27.192039 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:27.192005    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:44.111086 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:44.110982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:44.942325 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:44.942270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:47.854131 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:47.854086    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:47.948000 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:47.947966    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:48.511136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:48.511098    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:48.910235 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:48.910163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:50.202721 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:50.202686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:50.358892 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:50.358860    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:52.322020 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:52.321978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:52.420563 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:52.420531    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:54.512338 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:54.512275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:55.964909 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:55.964874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:57.525473 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:57.525438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:57.630804 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:57.630771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:58.650293 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.650246    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"]
Aug 04 11:40:58.650749 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.650573    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66445014-a469-4362-88cb-6ea74e8de74f" containerName="console"
Aug 04 11:40:58.650749 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.650584    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="66445014-a469-4362-88cb-6ea74e8de74f" containerName="console"
Aug 04 11:40:58.650749 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.650653    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="66445014-a469-4362-88cb-6ea74e8de74f" containerName="console"
Aug 04 11:40:58.654899 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.654877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.655384 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.655358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:40:58.657444 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.657421    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Aug 04 11:40:58.658620 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.658604    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nfnjq\""
Aug 04 11:40:58.658688 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.658658    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Aug 04 11:40:58.658688 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.658677    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 11:40:58.658770 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.658685    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 11:40:58.663611 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.663583    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"]
Aug 04 11:40:58.678490 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.678468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.678584 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.678498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g269c\" (UniqueName: \"kubernetes.io/projected/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-kube-api-access-g269c\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.678626 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.678580    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.778986 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.778944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.779127 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.778996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g269c\" (UniqueName: \"kubernetes.io/projected/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-kube-api-access-g269c\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.779127 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.779068    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.779694 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.779672    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.781227 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.781208    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.788014 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.787990    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g269c\" (UniqueName: \"kubernetes.io/projected/dee418d7-5d82-4a0e-8b0c-f2631cb3207b-kube-api-access-g269c\") pod \"kuadrant-console-plugin-598f86468b-57tn2\" (UID: \"dee418d7-5d82-4a0e-8b0c-f2631cb3207b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:58.965817 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:58.965735    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"
Aug 04 11:40:59.098162 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:59.098134    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2"]
Aug 04 11:40:59.099345 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:40:59.099314    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddee418d7_5d82_4a0e_8b0c_f2631cb3207b.slice/crio-c280f9c7c71a34cd86678d7cef39ad600ed568af2309972f4380ff3ee4b971c8 WatchSource:0}: Error finding container c280f9c7c71a34cd86678d7cef39ad600ed568af2309972f4380ff3ee4b971c8: Status 404 returned error can't find the container with id c280f9c7c71a34cd86678d7cef39ad600ed568af2309972f4380ff3ee4b971c8
Aug 04 11:40:59.157544 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:59.157508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2" event={"ID":"dee418d7-5d82-4a0e-8b0c-f2631cb3207b","Type":"ContainerStarted","Data":"c280f9c7c71a34cd86678d7cef39ad600ed568af2309972f4380ff3ee4b971c8"}
Aug 04 11:40:59.322100 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:40:59.322065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:03.722603 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:03.722573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:03.823194 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:03.823158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:03.923865 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:03.923787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:04.023439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:04.023408    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:04.174633 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:04.174551    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:04.188995 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:04.188965    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2" event={"ID":"dee418d7-5d82-4a0e-8b0c-f2631cb3207b","Type":"ContainerStarted","Data":"034a359a360fd2418bd92517f95eb5623ce2e518caa534f53fc6093e623fb1a3"}
Aug 04 11:41:04.211924 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:04.211872    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-57tn2" podStartSLOduration=1.658244523 podStartE2EDuration="6.211857399s" totalImagesPullingTime="4.553612876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:41:04.209762807 +0000 UTC m=+477.780675610" watchObservedRunningTime="2026-08-04 11:41:04.211857399 +0000 UTC m=+477.782770204"
Aug 04 11:41:04.274801 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:04.274770    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:09.669918 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:09.669882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:09.745318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:09.745264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:14.427909 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:14.427871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:14.528124 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:14.528093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:17.337781 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:17.337748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:17.398734 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:17.398704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:18.190152 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:18.190121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:18.244395 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:18.244368    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:22.996779 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:22.996742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:23.098058 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:23.098027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:23.395174 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:23.395144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:23.454531 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:23.454499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:23.825550 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:23.825522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:23.895863 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:23.895831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:29.116390 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:29.116360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:29.224623 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:29.224590    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:29.318083 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:29.318053    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:33.857663 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:33.857629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:33.961013 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:33.960976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:42.498838 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:42.498804    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:42.642772 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:42.642744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:47.289657 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:47.289621    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:47.409787 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:47.409760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:47.716765 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:47.716684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:48.108088 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.108057    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-kmpn5"]
Aug 04 11:41:48.127932 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.127905    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-kmpn5"]
Aug 04 11:41:48.128109 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.128010    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:48.130675 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.130650    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nw6zj\""
Aug 04 11:41:48.209595 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.209557    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hrhc\" (UniqueName: \"kubernetes.io/projected/4d0bd9b6-f24e-433a-8b44-2a906113f9d0-kube-api-access-9hrhc\") pod \"authorino-b54df94b4-kmpn5\" (UID: \"4d0bd9b6-f24e-433a-8b44-2a906113f9d0\") " pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:48.218141 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.218107    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xr9q4"]
Aug 04 11:41:48.221650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.221631    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:41:48.232478 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.232453    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xr9q4"]
Aug 04 11:41:48.260990 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.260961    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:48.310068 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.310029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k44m\" (UniqueName: \"kubernetes.io/projected/158cc900-7d5a-4324-ab65-3c26083e98b7-kube-api-access-2k44m\") pod \"authorino-c548b5d7d-xr9q4\" (UID: \"158cc900-7d5a-4324-ab65-3c26083e98b7\") " pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:41:48.310600 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.310106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hrhc\" (UniqueName: \"kubernetes.io/projected/4d0bd9b6-f24e-433a-8b44-2a906113f9d0-kube-api-access-9hrhc\") pod \"authorino-b54df94b4-kmpn5\" (UID: \"4d0bd9b6-f24e-433a-8b44-2a906113f9d0\") " pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:48.320191 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.320162    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hrhc\" (UniqueName: \"kubernetes.io/projected/4d0bd9b6-f24e-433a-8b44-2a906113f9d0-kube-api-access-9hrhc\") pod \"authorino-b54df94b4-kmpn5\" (UID: \"4d0bd9b6-f24e-433a-8b44-2a906113f9d0\") " pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:48.410597 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.410516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k44m\" (UniqueName: \"kubernetes.io/projected/158cc900-7d5a-4324-ab65-3c26083e98b7-kube-api-access-2k44m\") pod \"authorino-c548b5d7d-xr9q4\" (UID: \"158cc900-7d5a-4324-ab65-3c26083e98b7\") " pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:41:48.419271 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.419250    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k44m\" (UniqueName: \"kubernetes.io/projected/158cc900-7d5a-4324-ab65-3c26083e98b7-kube-api-access-2k44m\") pod \"authorino-c548b5d7d-xr9q4\" (UID: \"158cc900-7d5a-4324-ab65-3c26083e98b7\") " pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:41:48.437719 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.437691    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:48.532348 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.532310    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:41:48.569692 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.569666    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-kmpn5"]
Aug 04 11:41:48.571572 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:41:48.571542    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d0bd9b6_f24e_433a_8b44_2a906113f9d0.slice/crio-a49853f8d9898cbd67dfd3f20a5db3c874c461bb92468a81d3a076b814b89261 WatchSource:0}: Error finding container a49853f8d9898cbd67dfd3f20a5db3c874c461bb92468a81d3a076b814b89261: Status 404 returned error can't find the container with id a49853f8d9898cbd67dfd3f20a5db3c874c461bb92468a81d3a076b814b89261
Aug 04 11:41:48.660076 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:48.660053    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xr9q4"]
Aug 04 11:41:48.662060 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:41:48.661997    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod158cc900_7d5a_4324_ab65_3c26083e98b7.slice/crio-39461e262f20e6381ce102aa995f5ff10222d0c56e649721de9930075cabd1d8 WatchSource:0}: Error finding container 39461e262f20e6381ce102aa995f5ff10222d0c56e649721de9930075cabd1d8: Status 404 returned error can't find the container with id 39461e262f20e6381ce102aa995f5ff10222d0c56e649721de9930075cabd1d8
Aug 04 11:41:49.065946 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:49.065915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:41:49.404769 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:49.404661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xr9q4" event={"ID":"158cc900-7d5a-4324-ab65-3c26083e98b7","Type":"ContainerStarted","Data":"39461e262f20e6381ce102aa995f5ff10222d0c56e649721de9930075cabd1d8"}
Aug 04 11:41:49.406823 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:49.406759    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-kmpn5" event={"ID":"4d0bd9b6-f24e-433a-8b44-2a906113f9d0","Type":"ContainerStarted","Data":"a49853f8d9898cbd67dfd3f20a5db3c874c461bb92468a81d3a076b814b89261"}
Aug 04 11:41:52.424721 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:52.424679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xr9q4" event={"ID":"158cc900-7d5a-4324-ab65-3c26083e98b7","Type":"ContainerStarted","Data":"635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c"}
Aug 04 11:41:52.426049 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:52.426026    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-kmpn5" event={"ID":"4d0bd9b6-f24e-433a-8b44-2a906113f9d0","Type":"ContainerStarted","Data":"231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd"}
Aug 04 11:41:52.441017 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:52.440972    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-xr9q4" podStartSLOduration=1.5881813390000001 podStartE2EDuration="4.440958134s" totalImagesPullingTime="2.852776795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:41:52.440476458 +0000 UTC m=+526.011389262" watchObservedRunningTime="2026-08-04 11:41:52.440958134 +0000 UTC m=+526.011870938"
Aug 04 11:41:52.457139 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:52.457096    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-kmpn5" podStartSLOduration=1.500773929 podStartE2EDuration="4.457082766s" totalImagesPullingTime="2.956308837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:41:52.455701732 +0000 UTC m=+526.026614537" watchObservedRunningTime="2026-08-04 11:41:52.457082766 +0000 UTC m=+526.027995575"
Aug 04 11:41:52.468121 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:52.468093    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-kmpn5"]
Aug 04 11:41:54.435317 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:54.435241    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-kmpn5" podUID="4d0bd9b6-f24e-433a-8b44-2a906113f9d0" containerName="authorino" containerID="cri-o://231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd" gracePeriod=30
Aug 04 11:41:54.678586 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:54.678563    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:54.762289 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:54.762261    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hrhc\" (UniqueName: \"kubernetes.io/projected/4d0bd9b6-f24e-433a-8b44-2a906113f9d0-kube-api-access-9hrhc\") pod \"4d0bd9b6-f24e-433a-8b44-2a906113f9d0\" (UID: \"4d0bd9b6-f24e-433a-8b44-2a906113f9d0\") "
Aug 04 11:41:54.764409 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:54.764376    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0bd9b6-f24e-433a-8b44-2a906113f9d0-kube-api-access-9hrhc" (OuterVolumeSpecName: "kube-api-access-9hrhc") pod "4d0bd9b6-f24e-433a-8b44-2a906113f9d0" (UID: "4d0bd9b6-f24e-433a-8b44-2a906113f9d0"). InnerVolumeSpecName "kube-api-access-9hrhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:41:54.863083 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:54.863055    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hrhc\" (UniqueName: \"kubernetes.io/projected/4d0bd9b6-f24e-433a-8b44-2a906113f9d0-kube-api-access-9hrhc\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:41:55.440965 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.440936    2558 generic.go:356] "Generic (PLEG): container finished" podID="4d0bd9b6-f24e-433a-8b44-2a906113f9d0" containerID="231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd" exitCode=0
Aug 04 11:41:55.441373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.440995    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-kmpn5"
Aug 04 11:41:55.441373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.441031    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-kmpn5" event={"ID":"4d0bd9b6-f24e-433a-8b44-2a906113f9d0","Type":"ContainerDied","Data":"231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd"}
Aug 04 11:41:55.441373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.441075    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-kmpn5" event={"ID":"4d0bd9b6-f24e-433a-8b44-2a906113f9d0","Type":"ContainerDied","Data":"a49853f8d9898cbd67dfd3f20a5db3c874c461bb92468a81d3a076b814b89261"}
Aug 04 11:41:55.441373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.441092    2558 scope.go:117] "RemoveContainer" containerID="231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd"
Aug 04 11:41:55.449257 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.449243    2558 scope.go:117] "RemoveContainer" containerID="231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd"
Aug 04 11:41:55.449564 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:41:55.449542    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd\": container with ID starting with 231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd not found: ID does not exist" containerID="231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd"
Aug 04 11:41:55.449643 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.449570    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd"} err="failed to get container status \"231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd\": rpc error: code = NotFound desc = could not find container \"231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd\": container with ID starting with 231b6552a8b3af00267c2fe887e6ad302f8ac13342eab1929c6375e61a02e8bd not found: ID does not exist"
Aug 04 11:41:55.460198 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.460177    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-kmpn5"]
Aug 04 11:41:55.462670 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:55.462653    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-kmpn5"]
Aug 04 11:41:56.994762 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:41:56.994730    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d0bd9b6-f24e-433a-8b44-2a906113f9d0" path="/var/lib/kubelet/pods/4d0bd9b6-f24e-433a-8b44-2a906113f9d0/volumes"
Aug 04 11:42:05.751462 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:05.751371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:06.206042 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:06.206012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:06.318479 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:06.318444    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:17.701520 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.701482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:17.920155 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.920122    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-pzlgw"]
Aug 04 11:42:17.920467 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.920455    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0bd9b6-f24e-433a-8b44-2a906113f9d0" containerName="authorino"
Aug 04 11:42:17.920514 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.920469    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0bd9b6-f24e-433a-8b44-2a906113f9d0" containerName="authorino"
Aug 04 11:42:17.920557 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.920549    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0bd9b6-f24e-433a-8b44-2a906113f9d0" containerName="authorino"
Aug 04 11:42:17.926976 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.926949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:17.932414 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:17.932388    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-pzlgw"]
Aug 04 11:42:18.016875 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.016848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:18.059996 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.059962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57vnj\" (UniqueName: \"kubernetes.io/projected/75959718-62ac-45fb-ab83-236523822f1a-kube-api-access-57vnj\") pod \"authorino-5dc7ccb5f-pzlgw\" (UID: \"75959718-62ac-45fb-ab83-236523822f1a\") " pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:18.130701 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.130667    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-pzlgw"]
Aug 04 11:42:18.130943 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:42:18.130921    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-57vnj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw" podUID="75959718-62ac-45fb-ab83-236523822f1a"
Aug 04 11:42:18.160452 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.160418    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d5b655689-8wjqc"]
Aug 04 11:42:18.160751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.160733    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57vnj\" (UniqueName: \"kubernetes.io/projected/75959718-62ac-45fb-ab83-236523822f1a-kube-api-access-57vnj\") pod \"authorino-5dc7ccb5f-pzlgw\" (UID: \"75959718-62ac-45fb-ab83-236523822f1a\") " pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:18.164211 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.164194    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:18.175886 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.175861    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d5b655689-8wjqc"]
Aug 04 11:42:18.177884 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.177862    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57vnj\" (UniqueName: \"kubernetes.io/projected/75959718-62ac-45fb-ab83-236523822f1a-kube-api-access-57vnj\") pod \"authorino-5dc7ccb5f-pzlgw\" (UID: \"75959718-62ac-45fb-ab83-236523822f1a\") " pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:18.220573 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.220541    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d5b655689-8wjqc"]
Aug 04 11:42:18.220813 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:42:18.220792    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-c4p4j], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d5b655689-8wjqc" podUID="9f8f4394-7073-469d-8e04-06f4b5d0a17e"
Aug 04 11:42:18.250345 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.250310    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5446f4b69f-2vxwr"]
Aug 04 11:42:18.254148 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.254125    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.256672 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.256654    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 04 11:42:18.261143 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.261120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4p4j\" (UniqueName: \"kubernetes.io/projected/9f8f4394-7073-469d-8e04-06f4b5d0a17e-kube-api-access-c4p4j\") pod \"authorino-6d5b655689-8wjqc\" (UID: \"9f8f4394-7073-469d-8e04-06f4b5d0a17e\") " pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:18.261663 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.261639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5446f4b69f-2vxwr"]
Aug 04 11:42:18.361671 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.361589    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-tls-cert\") pod \"authorino-5446f4b69f-2vxwr\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") " pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.361671 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.361649    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4p4j\" (UniqueName: \"kubernetes.io/projected/9f8f4394-7073-469d-8e04-06f4b5d0a17e-kube-api-access-c4p4j\") pod \"authorino-6d5b655689-8wjqc\" (UID: \"9f8f4394-7073-469d-8e04-06f4b5d0a17e\") " pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:18.361859 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.361676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x5md\" (UniqueName: \"kubernetes.io/projected/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-kube-api-access-2x5md\") pod \"authorino-5446f4b69f-2vxwr\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") " pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.369402 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.369370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4p4j\" (UniqueName: \"kubernetes.io/projected/9f8f4394-7073-469d-8e04-06f4b5d0a17e-kube-api-access-c4p4j\") pod \"authorino-6d5b655689-8wjqc\" (UID: \"9f8f4394-7073-469d-8e04-06f4b5d0a17e\") " pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:18.462517 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.462475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-tls-cert\") pod \"authorino-5446f4b69f-2vxwr\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") " pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.462682 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.462537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x5md\" (UniqueName: \"kubernetes.io/projected/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-kube-api-access-2x5md\") pod \"authorino-5446f4b69f-2vxwr\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") " pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.464951 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.464920    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-tls-cert\") pod \"authorino-5446f4b69f-2vxwr\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") " pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.472117 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.472089    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x5md\" (UniqueName: \"kubernetes.io/projected/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-kube-api-access-2x5md\") pod \"authorino-5446f4b69f-2vxwr\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") " pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.545577 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.545545    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:18.545754 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.545670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:18.553213 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.553189    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:18.555746 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.555726    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:18.564437 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.564416    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:42:18.664733 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.664705    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4p4j\" (UniqueName: \"kubernetes.io/projected/9f8f4394-7073-469d-8e04-06f4b5d0a17e-kube-api-access-c4p4j\") pod \"9f8f4394-7073-469d-8e04-06f4b5d0a17e\" (UID: \"9f8f4394-7073-469d-8e04-06f4b5d0a17e\") "
Aug 04 11:42:18.664913 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.664764    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57vnj\" (UniqueName: \"kubernetes.io/projected/75959718-62ac-45fb-ab83-236523822f1a-kube-api-access-57vnj\") pod \"75959718-62ac-45fb-ab83-236523822f1a\" (UID: \"75959718-62ac-45fb-ab83-236523822f1a\") "
Aug 04 11:42:18.667029 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.666967    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75959718-62ac-45fb-ab83-236523822f1a-kube-api-access-57vnj" (OuterVolumeSpecName: "kube-api-access-57vnj") pod "75959718-62ac-45fb-ab83-236523822f1a" (UID: "75959718-62ac-45fb-ab83-236523822f1a"). InnerVolumeSpecName "kube-api-access-57vnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:42:18.667029 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.667004    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8f4394-7073-469d-8e04-06f4b5d0a17e-kube-api-access-c4p4j" (OuterVolumeSpecName: "kube-api-access-c4p4j") pod "9f8f4394-7073-469d-8e04-06f4b5d0a17e" (UID: "9f8f4394-7073-469d-8e04-06f4b5d0a17e"). InnerVolumeSpecName "kube-api-access-c4p4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:42:18.698641 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.698596    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5446f4b69f-2vxwr"]
Aug 04 11:42:18.700818 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:42:18.700790    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0f1d532_86b8_40c6_a2f0_dd4c2f384955.slice/crio-fdeda06dc54b4bcb962323fdbc33adde037ac6090ab5ee937bb3cae774bc3d3e WatchSource:0}: Error finding container fdeda06dc54b4bcb962323fdbc33adde037ac6090ab5ee937bb3cae774bc3d3e: Status 404 returned error can't find the container with id fdeda06dc54b4bcb962323fdbc33adde037ac6090ab5ee937bb3cae774bc3d3e
Aug 04 11:42:18.766202 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.766173    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4p4j\" (UniqueName: \"kubernetes.io/projected/9f8f4394-7073-469d-8e04-06f4b5d0a17e-kube-api-access-c4p4j\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:42:18.766202 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:18.766197    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57vnj\" (UniqueName: \"kubernetes.io/projected/75959718-62ac-45fb-ab83-236523822f1a-kube-api-access-57vnj\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:42:19.551711 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.551671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5446f4b69f-2vxwr" event={"ID":"b0f1d532-86b8-40c6-a2f0-dd4c2f384955","Type":"ContainerStarted","Data":"0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1"}
Aug 04 11:42:19.551711 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.551711    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b655689-8wjqc"
Aug 04 11:42:19.551948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.551723    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5446f4b69f-2vxwr" event={"ID":"b0f1d532-86b8-40c6-a2f0-dd4c2f384955","Type":"ContainerStarted","Data":"fdeda06dc54b4bcb962323fdbc33adde037ac6090ab5ee937bb3cae774bc3d3e"}
Aug 04 11:42:19.551948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.551695    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-pzlgw"
Aug 04 11:42:19.569373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.569326    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5446f4b69f-2vxwr" podStartSLOduration=1.168579701 podStartE2EDuration="1.569310692s" totalImagesPullingTime="400.730991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:42:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:42:19.567117106 +0000 UTC m=+553.138029920" watchObservedRunningTime="2026-08-04 11:42:19.569310692 +0000 UTC m=+553.140223524"
Aug 04 11:42:19.596711 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.596606    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xr9q4"]
Aug 04 11:42:19.596903 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.596834    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-xr9q4" podUID="158cc900-7d5a-4324-ab65-3c26083e98b7" containerName="authorino" containerID="cri-o://635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c" gracePeriod=30
Aug 04 11:42:19.599455 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.599421    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d5b655689-8wjqc"]
Aug 04 11:42:19.610739 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.610591    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d5b655689-8wjqc"]
Aug 04 11:42:19.633383 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.633341    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-pzlgw"]
Aug 04 11:42:19.635087 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.635067    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-pzlgw"]
Aug 04 11:42:19.837995 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.837975    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:42:19.977330 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.977300    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k44m\" (UniqueName: \"kubernetes.io/projected/158cc900-7d5a-4324-ab65-3c26083e98b7-kube-api-access-2k44m\") pod \"158cc900-7d5a-4324-ab65-3c26083e98b7\" (UID: \"158cc900-7d5a-4324-ab65-3c26083e98b7\") "
Aug 04 11:42:19.979321 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:19.979289    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158cc900-7d5a-4324-ab65-3c26083e98b7-kube-api-access-2k44m" (OuterVolumeSpecName: "kube-api-access-2k44m") pod "158cc900-7d5a-4324-ab65-3c26083e98b7" (UID: "158cc900-7d5a-4324-ab65-3c26083e98b7"). InnerVolumeSpecName "kube-api-access-2k44m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:42:20.077927 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.077846    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k44m\" (UniqueName: \"kubernetes.io/projected/158cc900-7d5a-4324-ab65-3c26083e98b7-kube-api-access-2k44m\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:42:20.557545 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.557515    2558 generic.go:356] "Generic (PLEG): container finished" podID="158cc900-7d5a-4324-ab65-3c26083e98b7" containerID="635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c" exitCode=0
Aug 04 11:42:20.557721 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.557552    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xr9q4"
Aug 04 11:42:20.557721 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.557597    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xr9q4" event={"ID":"158cc900-7d5a-4324-ab65-3c26083e98b7","Type":"ContainerDied","Data":"635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c"}
Aug 04 11:42:20.557721 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.557638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xr9q4" event={"ID":"158cc900-7d5a-4324-ab65-3c26083e98b7","Type":"ContainerDied","Data":"39461e262f20e6381ce102aa995f5ff10222d0c56e649721de9930075cabd1d8"}
Aug 04 11:42:20.557721 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.557655    2558 scope.go:117] "RemoveContainer" containerID="635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c"
Aug 04 11:42:20.566751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.566737    2558 scope.go:117] "RemoveContainer" containerID="635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c"
Aug 04 11:42:20.567003 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:42:20.566985    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c\": container with ID starting with 635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c not found: ID does not exist" containerID="635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c"
Aug 04 11:42:20.567067 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.567011    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c"} err="failed to get container status \"635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c\": rpc error: code = NotFound desc = could not find container \"635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c\": container with ID starting with 635926775d771c9d80447956fd97f377caa66ca63d64e120ce7fb7222e6ffd8c not found: ID does not exist"
Aug 04 11:42:20.581696 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.581674    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xr9q4"]
Aug 04 11:42:20.585872 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.585852    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xr9q4"]
Aug 04 11:42:20.993457 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.993366    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="158cc900-7d5a-4324-ab65-3c26083e98b7" path="/var/lib/kubelet/pods/158cc900-7d5a-4324-ab65-3c26083e98b7/volumes"
Aug 04 11:42:20.993948 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.993825    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75959718-62ac-45fb-ab83-236523822f1a" path="/var/lib/kubelet/pods/75959718-62ac-45fb-ab83-236523822f1a/volumes"
Aug 04 11:42:20.994099 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:20.994082    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8f4394-7073-469d-8e04-06f4b5d0a17e" path="/var/lib/kubelet/pods/9f8f4394-7073-469d-8e04-06f4b5d0a17e/volumes"
Aug 04 11:42:24.091525 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.091494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:24.177433 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.177404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:24.235126 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.235094    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:24.273360 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.273328    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-6qzhr"]
Aug 04 11:42:24.273678 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.273667    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158cc900-7d5a-4324-ab65-3c26083e98b7" containerName="authorino"
Aug 04 11:42:24.273729 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.273680    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="158cc900-7d5a-4324-ab65-3c26083e98b7" containerName="authorino"
Aug 04 11:42:24.273761 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.273748    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="158cc900-7d5a-4324-ab65-3c26083e98b7" containerName="authorino"
Aug 04 11:42:24.276505 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.276485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.280665 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.280646    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w46zc\""
Aug 04 11:42:24.280665 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.280643    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 11:42:24.280849 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.280685    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 11:42:24.285779 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.285755    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-6qzhr"]
Aug 04 11:42:24.335821 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.335792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:24.417669 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.417580    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ee3d590-f81b-466c-b74d-ba7ac2230a48-webhook-cert\") pod \"maas-controller-c67f8cc4c-6qzhr\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") " pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.417669 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.417628    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkz4p\" (UniqueName: \"kubernetes.io/projected/3ee3d590-f81b-466c-b74d-ba7ac2230a48-kube-api-access-lkz4p\") pod \"maas-controller-c67f8cc4c-6qzhr\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") " pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.518349 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.518315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ee3d590-f81b-466c-b74d-ba7ac2230a48-webhook-cert\") pod \"maas-controller-c67f8cc4c-6qzhr\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") " pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.518519 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.518362    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkz4p\" (UniqueName: \"kubernetes.io/projected/3ee3d590-f81b-466c-b74d-ba7ac2230a48-kube-api-access-lkz4p\") pod \"maas-controller-c67f8cc4c-6qzhr\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") " pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.520689 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.520662    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ee3d590-f81b-466c-b74d-ba7ac2230a48-webhook-cert\") pod \"maas-controller-c67f8cc4c-6qzhr\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") " pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.528337 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.528315    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkz4p\" (UniqueName: \"kubernetes.io/projected/3ee3d590-f81b-466c-b74d-ba7ac2230a48-kube-api-access-lkz4p\") pod \"maas-controller-c67f8cc4c-6qzhr\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") " pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.588086 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.588054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:24.713083 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:24.713058    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-6qzhr"]
Aug 04 11:42:24.714834 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:42:24.714810    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee3d590_f81b_466c_b74d_ba7ac2230a48.slice/crio-3d033205679f220e31733951673b8a38588969039961187f078bbe13a6953b81 WatchSource:0}: Error finding container 3d033205679f220e31733951673b8a38588969039961187f078bbe13a6953b81: Status 404 returned error can't find the container with id 3d033205679f220e31733951673b8a38588969039961187f078bbe13a6953b81
Aug 04 11:42:25.583978 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:25.583940    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr" event={"ID":"3ee3d590-f81b-466c-b74d-ba7ac2230a48","Type":"ContainerStarted","Data":"3d033205679f220e31733951673b8a38588969039961187f078bbe13a6953b81"}
Aug 04 11:42:27.013053 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:27.013009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:27.108416 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:27.108384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:27.596448 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:27.596412    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr" event={"ID":"3ee3d590-f81b-466c-b74d-ba7ac2230a48","Type":"ContainerStarted","Data":"f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9"}
Aug 04 11:42:27.596650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:27.596558    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:27.618023 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:27.617959    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr" podStartSLOduration=1.26277744 podStartE2EDuration="3.61793648s" totalImagesPullingTime="2.35515904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:42:27.616629175 +0000 UTC m=+561.187541978" watchObservedRunningTime="2026-08-04 11:42:27.61793648 +0000 UTC m=+561.188849284"
Aug 04 11:42:38.602314 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:38.602269    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:42:52.997304 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:52.997263    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:58.799126 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:58.799096    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:42:58.896816 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:42:58.896786    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:00.402109 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.402080    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:00.537378 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.537345    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"]
Aug 04 11:43:00.540860 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.540845    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.545412 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.545366    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9dpqw\""
Aug 04 11:43:00.545412 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.545400    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 11:43:00.545616 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.545437    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 04 11:43:00.545616 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.545366    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 04 11:43:00.551001 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.550978    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"]
Aug 04 11:43:00.563994 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.563974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:00.601993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.601967    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb828966-ec5d-481c-978c-1f1f9cd39626-openshift-service-ca\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.602108 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.602011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwzq\" (UniqueName: \"kubernetes.io/projected/bb828966-ec5d-481c-978c-1f1f9cd39626-kube-api-access-xbwzq\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.602108 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.602047    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb828966-ec5d-481c-978c-1f1f9cd39626-maas-api-tls\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.702538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.702442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb828966-ec5d-481c-978c-1f1f9cd39626-openshift-service-ca\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.702538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.702492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwzq\" (UniqueName: \"kubernetes.io/projected/bb828966-ec5d-481c-978c-1f1f9cd39626-kube-api-access-xbwzq\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.702538 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.702519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb828966-ec5d-481c-978c-1f1f9cd39626-maas-api-tls\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.703207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.703136    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb828966-ec5d-481c-978c-1f1f9cd39626-openshift-service-ca\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.705199 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.705180    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb828966-ec5d-481c-978c-1f1f9cd39626-maas-api-tls\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.712135 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.712108    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwzq\" (UniqueName: \"kubernetes.io/projected/bb828966-ec5d-481c-978c-1f1f9cd39626-kube-api-access-xbwzq\") pod \"maas-api-78547f99d8-g2tjv\" (UID: \"bb828966-ec5d-481c-978c-1f1f9cd39626\") " pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.852443 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.852406    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:00.876713 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.876678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:00.978625 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:00.978557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:01.000196 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:01.000156    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"]
Aug 04 11:43:01.002408 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:43:01.002383    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb828966_ec5d_481c_978c_1f1f9cd39626.slice/crio-addda90a9d8e8a5d23a9a5d7718e47de1286b917a8809de3da3d79c7eab61014 WatchSource:0}: Error finding container addda90a9d8e8a5d23a9a5d7718e47de1286b917a8809de3da3d79c7eab61014: Status 404 returned error can't find the container with id addda90a9d8e8a5d23a9a5d7718e47de1286b917a8809de3da3d79c7eab61014
Aug 04 11:43:01.758504 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:01.758471    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv" event={"ID":"bb828966-ec5d-481c-978c-1f1f9cd39626","Type":"ContainerStarted","Data":"addda90a9d8e8a5d23a9a5d7718e47de1286b917a8809de3da3d79c7eab61014"}
Aug 04 11:43:03.768544 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:03.768501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv" event={"ID":"bb828966-ec5d-481c-978c-1f1f9cd39626","Type":"ContainerStarted","Data":"19ff4588e3c5d368f7c248e502144bd887f34545f07c9216a6f7fcd8433d336e"}
Aug 04 11:43:03.768947 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:03.768676    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:03.787315 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:03.787250    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv" podStartSLOduration=1.97099829 podStartE2EDuration="3.787235245s" totalImagesPullingTime="1.816236955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:43:03.786622451 +0000 UTC m=+597.357535255" watchObservedRunningTime="2026-08-04 11:43:03.787235245 +0000 UTC m=+597.358148048"
Aug 04 11:43:06.881510 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:06.881472    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:43:06.884673 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:06.884654    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:43:09.778627 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:09.778600    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78547f99d8-g2tjv"
Aug 04 11:43:10.778263 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:10.778232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:10.879577 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:10.879545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:20.399919 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.399876    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"]
Aug 04 11:43:20.403354 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.403333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.406153 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.406120    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 11:43:20.407520 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.407497    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 11:43:20.407667 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.407536    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5k874\""
Aug 04 11:43:20.407667 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.407548    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Aug 04 11:43:20.414575 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.414553    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"]
Aug 04 11:43:20.432251 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.432224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:20.472238 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.472200    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dckz8\" (UniqueName: \"kubernetes.io/projected/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-kube-api-access-dckz8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.472439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.472257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.472439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.472325    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.472439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.472384    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.472439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.472421    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.472617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.472441    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573473 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dckz8\" (UniqueName: \"kubernetes.io/projected/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-kube-api-access-dckz8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573518    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.573937 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573915    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.574007 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.573957    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.574057 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.574005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.576612 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.576587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.576718 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.576703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.581169 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.581143    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dckz8\" (UniqueName: \"kubernetes.io/projected/54f8e3d3-f0bd-4f04-8637-03b9df69d2e3-kube-api-access-dckz8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km\" (UID: \"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.715217 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.715129    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:20.842299 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.842262    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"]
Aug 04 11:43:20.844375 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:43:20.844339    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f8e3d3_f0bd_4f04_8637_03b9df69d2e3.slice/crio-f391a09698713313222a244f34a3c8827a08468daab68d30b46337357fa80b60 WatchSource:0}: Error finding container f391a09698713313222a244f34a3c8827a08468daab68d30b46337357fa80b60: Status 404 returned error can't find the container with id f391a09698713313222a244f34a3c8827a08468daab68d30b46337357fa80b60
Aug 04 11:43:20.846221 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:20.846202    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 11:43:21.853653 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:21.853610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km" event={"ID":"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3","Type":"ContainerStarted","Data":"f391a09698713313222a244f34a3c8827a08468daab68d30b46337357fa80b60"}
Aug 04 11:43:22.861090 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:22.861053    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km" event={"ID":"54f8e3d3-f0bd-4f04-8637-03b9df69d2e3","Type":"ContainerStarted","Data":"9b570e00082727eb2c1d6b49bfd1184e15fc2c0bea12b0fccd66bcfc05429495"}
Aug 04 11:43:22.880356 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:22.880310    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km" podStartSLOduration=1.691419564 podStartE2EDuration="2.880294242s" totalImagesPullingTime="1.188874678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:43:22.878826318 +0000 UTC m=+616.449739140" watchObservedRunningTime="2026-08-04 11:43:22.880294242 +0000 UTC m=+616.451207038"
Aug 04 11:43:26.981151 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:26.981120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:27.082123 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:27.082085    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:30.716255 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:30.716178    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:30.716654 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:30.716384    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:30.728914 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:30.728894    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:30.912075 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:30.912048    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km"
Aug 04 11:43:48.100236 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:48.100199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:49.967215 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:49.967180    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:43:53.733641 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:43:53.733610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:44:00.726876 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:00.726844    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:44:18.481935 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.481877    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:44:18.583993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.583960    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:44:18.903917 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.903890    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"]
Aug 04 11:44:18.906394 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.906372    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:18.908979 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.908955    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Aug 04 11:44:18.918566 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.918547    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"]
Aug 04 11:44:18.969042 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.969009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:18.969042 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.969041    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbp6r\" (UniqueName: \"kubernetes.io/projected/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-kube-api-access-hbp6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:18.969306 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.969074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:18.969306 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.969136    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:18.969306 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.969181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:18.969306 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:18.969212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.001904 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.001865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:44:19.070147 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070335 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070335 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070332    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbp6r\" (UniqueName: \"kubernetes.io/projected/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-kube-api-access-hbp6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070501    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070571 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070510    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.070998 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.070976    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.071191 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.071018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.072624 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.072606    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.072963 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.072947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.079565 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.079545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbp6r\" (UniqueName: \"kubernetes.io/projected/b6350a18-01c0-4d03-ad4a-2465ef52a8d6-kube-api-access-hbp6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8x4f4\" (UID: \"b6350a18-01c0-4d03-ad4a-2465ef52a8d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.217492 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.217400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:19.349207 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:19.349184    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"]
Aug 04 11:44:19.351265 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:44:19.351234    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6350a18_01c0_4d03_ad4a_2465ef52a8d6.slice/crio-9077849d18c2273f951dfa55cf76f33e595ae1e79a8e3fbf66078324a69d4e7b WatchSource:0}: Error finding container 9077849d18c2273f951dfa55cf76f33e595ae1e79a8e3fbf66078324a69d4e7b: Status 404 returned error can't find the container with id 9077849d18c2273f951dfa55cf76f33e595ae1e79a8e3fbf66078324a69d4e7b
Aug 04 11:44:20.141096 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:20.141062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4" event={"ID":"b6350a18-01c0-4d03-ad4a-2465ef52a8d6","Type":"ContainerStarted","Data":"86a637d130d4f92abcc67465c4097d35ea12033f0d2a62d97c871facb1e1f310"}
Aug 04 11:44:20.141549 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:20.141116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4" event={"ID":"b6350a18-01c0-4d03-ad4a-2465ef52a8d6","Type":"ContainerStarted","Data":"9077849d18c2273f951dfa55cf76f33e595ae1e79a8e3fbf66078324a69d4e7b"}
Aug 04 11:44:20.159308 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:20.159247    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4" podStartSLOduration=1.9737832370000001 podStartE2EDuration="2.159231586s" totalImagesPullingTime="185.448349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:44:20.158619628 +0000 UTC m=+673.729532423" watchObservedRunningTime="2026-08-04 11:44:20.159231586 +0000 UTC m=+673.730144447"
Aug 04 11:44:27.099860 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:27.099822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:44:29.218035 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:29.218001    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:29.218457 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:29.218057    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:29.230433 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:29.230413    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:44:30.198346 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:44:30.198310    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8x4f4"
Aug 04 11:45:00.148399 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.148309    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"]
Aug 04 11:45:00.151526 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.151504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"
Aug 04 11:45:00.154562 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.154531    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 11:45:00.159349 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.159320    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"]
Aug 04 11:45:00.163996 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.163974    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"
Aug 04 11:45:00.299357 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.299331    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"]
Aug 04 11:45:00.301626 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:45:00.301595    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d22ade_8a2e_4556_9a43_893b4d89a49c.slice/crio-03b23db98b24a478ed62faf1f954181fd899e7dfdb086ab8c56070098d53d3fa WatchSource:0}: Error finding container 03b23db98b24a478ed62faf1f954181fd899e7dfdb086ab8c56070098d53d3fa: Status 404 returned error can't find the container with id 03b23db98b24a478ed62faf1f954181fd899e7dfdb086ab8c56070098d53d3fa
Aug 04 11:45:00.337119 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:00.337088    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n" event={"ID":"50d22ade-8a2e-4556-9a43-893b4d89a49c","Type":"ContainerStarted","Data":"03b23db98b24a478ed62faf1f954181fd899e7dfdb086ab8c56070098d53d3fa"}
Aug 04 11:45:13.913739 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:13.913705    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:14.008525 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:14.008491    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:30.484024 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:30.483931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n" event={"ID":"50d22ade-8a2e-4556-9a43-893b4d89a49c","Type":"ContainerStarted","Data":"e38d018de62c519a221d285a7f709c88abca650410c8f3a99cd6112b3f72335d"}
Aug 04 11:45:30.501332 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:30.501269    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n" podStartSLOduration=0.854196462 podStartE2EDuration="30.501254683s" totalImagesPullingTime="29.647058221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:45:30.499787554 +0000 UTC m=+744.070700358" watchObservedRunningTime="2026-08-04 11:45:30.501254683 +0000 UTC m=+744.072167487"
Aug 04 11:45:31.489591 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:31.489557    2558 generic.go:356] "Generic (PLEG): container finished" podID="50d22ade-8a2e-4556-9a43-893b4d89a49c" containerID="e38d018de62c519a221d285a7f709c88abca650410c8f3a99cd6112b3f72335d" exitCode=0
Aug 04 11:45:31.489987 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:31.489633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n" event={"ID":"50d22ade-8a2e-4556-9a43-893b4d89a49c","Type":"ContainerDied","Data":"e38d018de62c519a221d285a7f709c88abca650410c8f3a99cd6112b3f72335d"}
Aug 04 11:45:32.633332 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:32.633304    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"
Aug 04 11:45:33.500931 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:33.500897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n" event={"ID":"50d22ade-8a2e-4556-9a43-893b4d89a49c","Type":"ContainerDied","Data":"03b23db98b24a478ed62faf1f954181fd899e7dfdb086ab8c56070098d53d3fa"}
Aug 04 11:45:33.501077 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:33.500946    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03b23db98b24a478ed62faf1f954181fd899e7dfdb086ab8c56070098d53d3fa"
Aug 04 11:45:33.501077 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:33.500907    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"
Aug 04 11:45:48.560655 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:48.560619    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:48.660492 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:48.660460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:49.472182 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.472147    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:49.572269 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.572234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:49.614320 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.614270    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84dc45b674-lq5kc"]
Aug 04 11:45:49.614731 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.614713    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d22ade-8a2e-4556-9a43-893b4d89a49c" containerName="cleanup"
Aug 04 11:45:49.614822 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.614734    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d22ade-8a2e-4556-9a43-893b4d89a49c" containerName="cleanup"
Aug 04 11:45:49.614878 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.614829    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d22ade-8a2e-4556-9a43-893b4d89a49c" containerName="cleanup"
Aug 04 11:45:49.622028 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.621998    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.625880 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.625851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84dc45b674-lq5kc"]
Aug 04 11:45:49.642037 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.642012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqbxp\" (UniqueName: \"kubernetes.io/projected/4a39e227-f929-430c-b243-b12973d5576a-kube-api-access-rqbxp\") pod \"authorino-84dc45b674-lq5kc\" (UID: \"4a39e227-f929-430c-b243-b12973d5576a\") " pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.642169 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.642051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4a39e227-f929-430c-b243-b12973d5576a-tls-cert\") pod \"authorino-84dc45b674-lq5kc\" (UID: \"4a39e227-f929-430c-b243-b12973d5576a\") " pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.743138 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.743050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqbxp\" (UniqueName: \"kubernetes.io/projected/4a39e227-f929-430c-b243-b12973d5576a-kube-api-access-rqbxp\") pod \"authorino-84dc45b674-lq5kc\" (UID: \"4a39e227-f929-430c-b243-b12973d5576a\") " pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.743138 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.743095    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4a39e227-f929-430c-b243-b12973d5576a-tls-cert\") pod \"authorino-84dc45b674-lq5kc\" (UID: \"4a39e227-f929-430c-b243-b12973d5576a\") " pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.745534 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.745510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4a39e227-f929-430c-b243-b12973d5576a-tls-cert\") pod \"authorino-84dc45b674-lq5kc\" (UID: \"4a39e227-f929-430c-b243-b12973d5576a\") " pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.757093 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.757056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqbxp\" (UniqueName: \"kubernetes.io/projected/4a39e227-f929-430c-b243-b12973d5576a-kube-api-access-rqbxp\") pod \"authorino-84dc45b674-lq5kc\" (UID: \"4a39e227-f929-430c-b243-b12973d5576a\") " pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:49.933447 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:49.933408    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dc45b674-lq5kc"
Aug 04 11:45:50.070740 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:50.068859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84dc45b674-lq5kc"]
Aug 04 11:45:50.073492 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:45:50.073464    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a39e227_f929_430c_b243_b12973d5576a.slice/crio-641b09d17961dd668fa615cb13464e09debe8a91fb2df9b4a08c74f8d4b51574 WatchSource:0}: Error finding container 641b09d17961dd668fa615cb13464e09debe8a91fb2df9b4a08c74f8d4b51574: Status 404 returned error can't find the container with id 641b09d17961dd668fa615cb13464e09debe8a91fb2df9b4a08c74f8d4b51574
Aug 04 11:45:50.583457 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:50.583425    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84dc45b674-lq5kc" event={"ID":"4a39e227-f929-430c-b243-b12973d5576a","Type":"ContainerStarted","Data":"641b09d17961dd668fa615cb13464e09debe8a91fb2df9b4a08c74f8d4b51574"}
Aug 04 11:45:51.591323 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.591288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84dc45b674-lq5kc" event={"ID":"4a39e227-f929-430c-b243-b12973d5576a","Type":"ContainerStarted","Data":"86a333792392fa30a6130d802054ff8737c8e4f1b3ebf0a4996689655cc97895"}
Aug 04 11:45:51.613124 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.613070    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84dc45b674-lq5kc" podStartSLOduration=1.787615325 podStartE2EDuration="2.613053252s" totalImagesPullingTime="825.437927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:45:51.611828991 +0000 UTC m=+765.182741795" watchObservedRunningTime="2026-08-04 11:45:51.613053252 +0000 UTC m=+765.183966055"
Aug 04 11:45:51.663944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.663909    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5446f4b69f-2vxwr"]
Aug 04 11:45:51.664169 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.664132    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5446f4b69f-2vxwr" podUID="b0f1d532-86b8-40c6-a2f0-dd4c2f384955" containerName="authorino" containerID="cri-o://0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1" gracePeriod=30
Aug 04 11:45:51.931548 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.931522    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:45:51.962753 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.962722    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x5md\" (UniqueName: \"kubernetes.io/projected/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-kube-api-access-2x5md\") pod \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") "
Aug 04 11:45:51.962753 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.962759    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-tls-cert\") pod \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\" (UID: \"b0f1d532-86b8-40c6-a2f0-dd4c2f384955\") "
Aug 04 11:45:51.965661 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.965626    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-kube-api-access-2x5md" (OuterVolumeSpecName: "kube-api-access-2x5md") pod "b0f1d532-86b8-40c6-a2f0-dd4c2f384955" (UID: "b0f1d532-86b8-40c6-a2f0-dd4c2f384955"). InnerVolumeSpecName "kube-api-access-2x5md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:45:51.973753 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:51.973728    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b0f1d532-86b8-40c6-a2f0-dd4c2f384955" (UID: "b0f1d532-86b8-40c6-a2f0-dd4c2f384955"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:45:52.064121 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.064087    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x5md\" (UniqueName: \"kubernetes.io/projected/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-kube-api-access-2x5md\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:45:52.064121 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.064115    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0f1d532-86b8-40c6-a2f0-dd4c2f384955-tls-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:45:52.226462 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.226384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:52.314218 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.314190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:52.597053 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.597020    2558 generic.go:356] "Generic (PLEG): container finished" podID="b0f1d532-86b8-40c6-a2f0-dd4c2f384955" containerID="0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1" exitCode=0
Aug 04 11:45:52.597472 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.597079    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5446f4b69f-2vxwr"
Aug 04 11:45:52.597472 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.597109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5446f4b69f-2vxwr" event={"ID":"b0f1d532-86b8-40c6-a2f0-dd4c2f384955","Type":"ContainerDied","Data":"0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1"}
Aug 04 11:45:52.597472 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.597151    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5446f4b69f-2vxwr" event={"ID":"b0f1d532-86b8-40c6-a2f0-dd4c2f384955","Type":"ContainerDied","Data":"fdeda06dc54b4bcb962323fdbc33adde037ac6090ab5ee937bb3cae774bc3d3e"}
Aug 04 11:45:52.597472 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.597170    2558 scope.go:117] "RemoveContainer" containerID="0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1"
Aug 04 11:45:52.608058 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.608032    2558 scope.go:117] "RemoveContainer" containerID="0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1"
Aug 04 11:45:52.608562 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:45:52.608525    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1\": container with ID starting with 0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1 not found: ID does not exist" containerID="0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1"
Aug 04 11:45:52.608677 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.608573    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1"} err="failed to get container status \"0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1\": rpc error: code = NotFound desc = could not find container \"0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1\": container with ID starting with 0169441e1870c1edf67c53ea70b292dce6b8a716cfd2aa508f6ba1d9cf7711e1 not found: ID does not exist"
Aug 04 11:45:52.628530 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.628503    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5446f4b69f-2vxwr"]
Aug 04 11:45:52.630911 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.630888    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5446f4b69f-2vxwr"]
Aug 04 11:45:52.994309 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:52.994219    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f1d532-86b8-40c6-a2f0-dd4c2f384955" path="/var/lib/kubelet/pods/b0f1d532-86b8-40c6-a2f0-dd4c2f384955/volumes"
Aug 04 11:45:53.302396 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:53.302359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:45:53.410174 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:45:53.410129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:46:38.751906 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:38.751811    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-6qzhr"]
Aug 04 11:46:38.752495 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:38.752124    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr" podUID="3ee3d590-f81b-466c-b74d-ba7ac2230a48" containerName="manager" containerID="cri-o://f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9" gracePeriod=30
Aug 04 11:46:38.997860 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:38.997836    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:46:39.172727 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.172701    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ee3d590-f81b-466c-b74d-ba7ac2230a48-webhook-cert\") pod \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") "
Aug 04 11:46:39.172888 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.172760    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkz4p\" (UniqueName: \"kubernetes.io/projected/3ee3d590-f81b-466c-b74d-ba7ac2230a48-kube-api-access-lkz4p\") pod \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\" (UID: \"3ee3d590-f81b-466c-b74d-ba7ac2230a48\") "
Aug 04 11:46:39.174747 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.174714    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ee3d590-f81b-466c-b74d-ba7ac2230a48-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3ee3d590-f81b-466c-b74d-ba7ac2230a48" (UID: "3ee3d590-f81b-466c-b74d-ba7ac2230a48"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:46:39.174861 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.174757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee3d590-f81b-466c-b74d-ba7ac2230a48-kube-api-access-lkz4p" (OuterVolumeSpecName: "kube-api-access-lkz4p") pod "3ee3d590-f81b-466c-b74d-ba7ac2230a48" (UID: "3ee3d590-f81b-466c-b74d-ba7ac2230a48"). InnerVolumeSpecName "kube-api-access-lkz4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:46:39.274236 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.274204    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkz4p\" (UniqueName: \"kubernetes.io/projected/3ee3d590-f81b-466c-b74d-ba7ac2230a48-kube-api-access-lkz4p\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:46:39.274236 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.274229    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ee3d590-f81b-466c-b74d-ba7ac2230a48-webhook-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:46:39.836532 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.836502    2558 generic.go:356] "Generic (PLEG): container finished" podID="3ee3d590-f81b-466c-b74d-ba7ac2230a48" containerID="f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9" exitCode=0
Aug 04 11:46:39.836944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.836548    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr"
Aug 04 11:46:39.836944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.836592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr" event={"ID":"3ee3d590-f81b-466c-b74d-ba7ac2230a48","Type":"ContainerDied","Data":"f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9"}
Aug 04 11:46:39.836944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.836646    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-6qzhr" event={"ID":"3ee3d590-f81b-466c-b74d-ba7ac2230a48","Type":"ContainerDied","Data":"3d033205679f220e31733951673b8a38588969039961187f078bbe13a6953b81"}
Aug 04 11:46:39.836944 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.836669    2558 scope.go:117] "RemoveContainer" containerID="f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9"
Aug 04 11:46:39.845909 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.845891    2558 scope.go:117] "RemoveContainer" containerID="f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9"
Aug 04 11:46:39.846155 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:46:39.846136    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9\": container with ID starting with f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9 not found: ID does not exist" containerID="f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9"
Aug 04 11:46:39.846221 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.846163    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9"} err="failed to get container status \"f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9\": rpc error: code = NotFound desc = could not find container \"f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9\": container with ID starting with f9eac39661e6920ce7c62bc64cebb357e07a6cd1041d4531ae63486896d895e9 not found: ID does not exist"
Aug 04 11:46:39.859344 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.859320    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-6qzhr"]
Aug 04 11:46:39.864364 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:39.864345    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-6qzhr"]
Aug 04 11:46:40.995601 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:46:40.995569    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ee3d590-f81b-466c-b74d-ba7ac2230a48" path="/var/lib/kubelet/pods/3ee3d590-f81b-466c-b74d-ba7ac2230a48/volumes"
Aug 04 11:47:40.645154 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645121    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-27qx6"]
Aug 04 11:47:40.645650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645561    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f1d532-86b8-40c6-a2f0-dd4c2f384955" containerName="authorino"
Aug 04 11:47:40.645650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645574    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f1d532-86b8-40c6-a2f0-dd4c2f384955" containerName="authorino"
Aug 04 11:47:40.645650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645582    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ee3d590-f81b-466c-b74d-ba7ac2230a48" containerName="manager"
Aug 04 11:47:40.645650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645587    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ee3d590-f81b-466c-b74d-ba7ac2230a48" containerName="manager"
Aug 04 11:47:40.645650 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645648    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ee3d590-f81b-466c-b74d-ba7ac2230a48" containerName="manager"
Aug 04 11:47:40.645812 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.645658    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f1d532-86b8-40c6-a2f0-dd4c2f384955" containerName="authorino"
Aug 04 11:47:40.648505 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.648490    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.652696 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.652667    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 11:47:40.652854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.652718    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 11:47:40.652854 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.652724    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w46zc\""
Aug 04 11:47:40.659531 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.659510    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-27qx6"]
Aug 04 11:47:40.678364 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.678339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906b7b38-35ed-4d86-8d14-227363e855c0-webhook-cert\") pod \"maas-controller-c67f8cc4c-27qx6\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") " pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.678512 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.678366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzchm\" (UniqueName: \"kubernetes.io/projected/906b7b38-35ed-4d86-8d14-227363e855c0-kube-api-access-xzchm\") pod \"maas-controller-c67f8cc4c-27qx6\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") " pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.779166 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.779135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906b7b38-35ed-4d86-8d14-227363e855c0-webhook-cert\") pod \"maas-controller-c67f8cc4c-27qx6\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") " pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.779166 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.779168    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzchm\" (UniqueName: \"kubernetes.io/projected/906b7b38-35ed-4d86-8d14-227363e855c0-kube-api-access-xzchm\") pod \"maas-controller-c67f8cc4c-27qx6\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") " pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.781569 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.781552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906b7b38-35ed-4d86-8d14-227363e855c0-webhook-cert\") pod \"maas-controller-c67f8cc4c-27qx6\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") " pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.788099 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.788074    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzchm\" (UniqueName: \"kubernetes.io/projected/906b7b38-35ed-4d86-8d14-227363e855c0-kube-api-access-xzchm\") pod \"maas-controller-c67f8cc4c-27qx6\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") " pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:40.959976 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:40.959894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:41.299805 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:41.299782    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-27qx6"]
Aug 04 11:47:41.301395 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:47:41.301369    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906b7b38_35ed_4d86_8d14_227363e855c0.slice/crio-fc15043e6d05c4033249e5c4d64bc4b7103b73a1a802ebd70cec742a96faa6d8 WatchSource:0}: Error finding container fc15043e6d05c4033249e5c4d64bc4b7103b73a1a802ebd70cec742a96faa6d8: Status 404 returned error can't find the container with id fc15043e6d05c4033249e5c4d64bc4b7103b73a1a802ebd70cec742a96faa6d8
Aug 04 11:47:42.144949 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:42.144915    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-27qx6" event={"ID":"906b7b38-35ed-4d86-8d14-227363e855c0","Type":"ContainerStarted","Data":"4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca"}
Aug 04 11:47:42.145373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:42.144963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-27qx6" event={"ID":"906b7b38-35ed-4d86-8d14-227363e855c0","Type":"ContainerStarted","Data":"fc15043e6d05c4033249e5c4d64bc4b7103b73a1a802ebd70cec742a96faa6d8"}
Aug 04 11:47:42.145373 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:42.145112    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:42.164963 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:42.164910    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c67f8cc4c-27qx6" podStartSLOduration=1.434882966 podStartE2EDuration="2.164894438s" totalImagesPullingTime="730.011472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:47:42.161818588 +0000 UTC m=+875.732731391" watchObservedRunningTime="2026-08-04 11:47:42.164894438 +0000 UTC m=+875.735807291"
Aug 04 11:47:53.152201 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:53.152172    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:57.776230 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:57.776190    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-27qx6"]
Aug 04 11:47:57.776726 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:57.776408    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c67f8cc4c-27qx6" podUID="906b7b38-35ed-4d86-8d14-227363e855c0" containerName="manager" containerID="cri-o://4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca" gracePeriod=30
Aug 04 11:47:57.814147 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:57.814119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:47:58.035945 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.035883    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:58.122302 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.122255    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzchm\" (UniqueName: \"kubernetes.io/projected/906b7b38-35ed-4d86-8d14-227363e855c0-kube-api-access-xzchm\") pod \"906b7b38-35ed-4d86-8d14-227363e855c0\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") "
Aug 04 11:47:58.122464 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.122422    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906b7b38-35ed-4d86-8d14-227363e855c0-webhook-cert\") pod \"906b7b38-35ed-4d86-8d14-227363e855c0\" (UID: \"906b7b38-35ed-4d86-8d14-227363e855c0\") "
Aug 04 11:47:58.124353 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.124323    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/906b7b38-35ed-4d86-8d14-227363e855c0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "906b7b38-35ed-4d86-8d14-227363e855c0" (UID: "906b7b38-35ed-4d86-8d14-227363e855c0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:47:58.124353 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.124334    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906b7b38-35ed-4d86-8d14-227363e855c0-kube-api-access-xzchm" (OuterVolumeSpecName: "kube-api-access-xzchm") pod "906b7b38-35ed-4d86-8d14-227363e855c0" (UID: "906b7b38-35ed-4d86-8d14-227363e855c0"). InnerVolumeSpecName "kube-api-access-xzchm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:47:58.223173 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223092    2558 generic.go:356] "Generic (PLEG): container finished" podID="906b7b38-35ed-4d86-8d14-227363e855c0" containerID="4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca" exitCode=0
Aug 04 11:47:58.223173 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223117    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906b7b38-35ed-4d86-8d14-227363e855c0-webhook-cert\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:47:58.223173 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223135    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzchm\" (UniqueName: \"kubernetes.io/projected/906b7b38-35ed-4d86-8d14-227363e855c0-kube-api-access-xzchm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:47:58.223173 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223143    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-27qx6"
Aug 04 11:47:58.223483 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223172    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-27qx6" event={"ID":"906b7b38-35ed-4d86-8d14-227363e855c0","Type":"ContainerDied","Data":"4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca"}
Aug 04 11:47:58.223483 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-27qx6" event={"ID":"906b7b38-35ed-4d86-8d14-227363e855c0","Type":"ContainerDied","Data":"fc15043e6d05c4033249e5c4d64bc4b7103b73a1a802ebd70cec742a96faa6d8"}
Aug 04 11:47:58.223483 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.223248    2558 scope.go:117] "RemoveContainer" containerID="4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca"
Aug 04 11:47:58.232013 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.231988    2558 scope.go:117] "RemoveContainer" containerID="4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca"
Aug 04 11:47:58.232258 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:47:58.232240    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca\": container with ID starting with 4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca not found: ID does not exist" containerID="4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca"
Aug 04 11:47:58.232335 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.232266    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca"} err="failed to get container status \"4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca\": rpc error: code = NotFound desc = could not find container \"4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca\": container with ID starting with 4c43150dda723b3e9e571a36eed7e2bf6b4d264e4a51dd7d3820568ab51f1bca not found: ID does not exist"
Aug 04 11:47:58.246156 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.246133    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-27qx6"]
Aug 04 11:47:58.250809 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.250786    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-27qx6"]
Aug 04 11:47:58.953745 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.953713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:47:58.994366 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:58.994327    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906b7b38-35ed-4d86-8d14-227363e855c0" path="/var/lib/kubelet/pods/906b7b38-35ed-4d86-8d14-227363e855c0/volumes"
Aug 04 11:47:59.012924 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.012892    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-bh6lm"]
Aug 04 11:47:59.013318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.013272    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906b7b38-35ed-4d86-8d14-227363e855c0" containerName="manager"
Aug 04 11:47:59.013318 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.013306    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="906b7b38-35ed-4d86-8d14-227363e855c0" containerName="manager"
Aug 04 11:47:59.013499 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.013435    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="906b7b38-35ed-4d86-8d14-227363e855c0" containerName="manager"
Aug 04 11:47:59.018362 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.018345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.021154 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.021135    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 11:47:59.022693 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.022655    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 11:47:59.022800 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.022714    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w46zc\""
Aug 04 11:47:59.026474 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.026455    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-bh6lm"]
Aug 04 11:47:59.130354 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.130314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ae726b4-46f1-4d8d-887d-97079ffa5dd0-webhook-cert\") pod \"maas-controller-c67f8cc4c-bh6lm\" (UID: \"1ae726b4-46f1-4d8d-887d-97079ffa5dd0\") " pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.130561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.130467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qssmt\" (UniqueName: \"kubernetes.io/projected/1ae726b4-46f1-4d8d-887d-97079ffa5dd0-kube-api-access-qssmt\") pod \"maas-controller-c67f8cc4c-bh6lm\" (UID: \"1ae726b4-46f1-4d8d-887d-97079ffa5dd0\") " pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.230993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.230906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qssmt\" (UniqueName: \"kubernetes.io/projected/1ae726b4-46f1-4d8d-887d-97079ffa5dd0-kube-api-access-qssmt\") pod \"maas-controller-c67f8cc4c-bh6lm\" (UID: \"1ae726b4-46f1-4d8d-887d-97079ffa5dd0\") " pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.230993 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.230964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ae726b4-46f1-4d8d-887d-97079ffa5dd0-webhook-cert\") pod \"maas-controller-c67f8cc4c-bh6lm\" (UID: \"1ae726b4-46f1-4d8d-887d-97079ffa5dd0\") " pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.233460 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.233433    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ae726b4-46f1-4d8d-887d-97079ffa5dd0-webhook-cert\") pod \"maas-controller-c67f8cc4c-bh6lm\" (UID: \"1ae726b4-46f1-4d8d-887d-97079ffa5dd0\") " pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.239297 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.239264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qssmt\" (UniqueName: \"kubernetes.io/projected/1ae726b4-46f1-4d8d-887d-97079ffa5dd0-kube-api-access-qssmt\") pod \"maas-controller-c67f8cc4c-bh6lm\" (UID: \"1ae726b4-46f1-4d8d-887d-97079ffa5dd0\") " pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.329427 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.329396    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:47:59.470015 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:47:59.469983    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c67f8cc4c-bh6lm"]
Aug 04 11:47:59.473104 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:47:59.473077    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae726b4_46f1_4d8d_887d_97079ffa5dd0.slice/crio-1b8bcaf64a3388dc27cc4529426884ba4268739bc41cee944e5d5dd10a5e6904 WatchSource:0}: Error finding container 1b8bcaf64a3388dc27cc4529426884ba4268739bc41cee944e5d5dd10a5e6904: Status 404 returned error can't find the container with id 1b8bcaf64a3388dc27cc4529426884ba4268739bc41cee944e5d5dd10a5e6904
Aug 04 11:48:00.238193 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:00.238140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm" event={"ID":"1ae726b4-46f1-4d8d-887d-97079ffa5dd0","Type":"ContainerStarted","Data":"29c05cfc01255306e8c76d313c6b83dfb100b9cea12b4cefd61c2d9a91ca61e1"}
Aug 04 11:48:00.238612 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:00.238207    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm" event={"ID":"1ae726b4-46f1-4d8d-887d-97079ffa5dd0","Type":"ContainerStarted","Data":"1b8bcaf64a3388dc27cc4529426884ba4268739bc41cee944e5d5dd10a5e6904"}
Aug 04 11:48:00.238612 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:00.238367    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:48:00.258009 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:00.257963    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm" podStartSLOduration=1.736009053 podStartE2EDuration="2.257943473s" totalImagesPullingTime="521.93442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:48:00.256300627 +0000 UTC m=+893.827213433" watchObservedRunningTime="2026-08-04 11:48:00.257943473 +0000 UTC m=+893.828856305"
Aug 04 11:48:06.920030 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:06.920006    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:48:06.920465 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:06.920006    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:48:11.245637 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:11.245607    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c67f8cc4c-bh6lm"
Aug 04 11:48:59.464614 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:48:59.464584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:01.865630 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:01.865598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:05.479503 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:05.479471    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:05.592239 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:05.592206    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:05.678386 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:05.678347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:05.816387 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:05.816357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:11.676136 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:11.676097    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:11.777024 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:11.776978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:28.923866 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:28.923784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:49:54.914576 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:49:54.914540    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:52:05.876291 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:52:05.876242    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:53:06.954609 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:53:06.954582    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:53:06.958324 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:53:06.958301    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:53:22.551559 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:53:22.551519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:55:00.012726 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:55:00.012694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:58:05.429641 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:05.429604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:58:06.983751 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:06.983722    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:58:06.989895 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:06.989869    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 11:58:20.129565 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:20.129534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:58:34.204508 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.204432    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"]
Aug 04 11:58:34.207757 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.207741    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.210517 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.210498    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d25909fe-kserve-self-signed-certs\""
Aug 04 11:58:34.225464 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.225439    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"]
Aug 04 11:58:34.230980 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.230957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:58:34.347670 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.347634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-home\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.347670 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.347667    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-model-cache\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.347890 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.347691    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-dshm\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.347890 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.347826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrzrs\" (UniqueName: \"kubernetes.io/projected/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-kube-api-access-qrzrs\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.347890 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.347859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tmp-dir\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.348006 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.347911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tls-certs\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.448980 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.448936    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-home\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.448980 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.448985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-model-cache\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449244 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-dshm\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449244 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449114    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrzrs\" (UniqueName: \"kubernetes.io/projected/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-kube-api-access-qrzrs\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449244 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tmp-dir\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449244 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tls-certs\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449496 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449342    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-home\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449496 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-model-cache\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.449606 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.449534    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tmp-dir\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.451444 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.451424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-dshm\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.451561 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.451535    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tls-certs\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.457325 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.457265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrzrs\" (UniqueName: \"kubernetes.io/projected/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-kube-api-access-qrzrs\") pod \"e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") " pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.516923 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.516897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:34.644715 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:34.644684    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"]
Aug 04 11:58:34.647609 ip-10-0-129-11 kubenswrapper[2558]: W0804 11:58:34.647581    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c2b3b37_7cfd_46f1_99c9_65753d9805fc.slice/crio-001b641200ed007759d6a9800013c0c87aaadfc10f1858aabf6e92e9168f9438 WatchSource:0}: Error finding container 001b641200ed007759d6a9800013c0c87aaadfc10f1858aabf6e92e9168f9438: Status 404 returned error can't find the container with id 001b641200ed007759d6a9800013c0c87aaadfc10f1858aabf6e92e9168f9438
Aug 04 11:58:35.364716 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:35.364682    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472" event={"ID":"4c2b3b37-7cfd-46f1-99c9-65753d9805fc","Type":"ContainerStarted","Data":"67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1"}
Aug 04 11:58:35.364716 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:35.364722    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472" event={"ID":"4c2b3b37-7cfd-46f1-99c9-65753d9805fc","Type":"ContainerStarted","Data":"001b641200ed007759d6a9800013c0c87aaadfc10f1858aabf6e92e9168f9438"}
Aug 04 11:58:35.384928 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:35.384869    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472" podStartSLOduration=1.384849636 podStartE2EDuration="1.384849636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:58:35.384032203 +0000 UTC m=+1528.954945007" watchObservedRunningTime="2026-08-04 11:58:35.384849636 +0000 UTC m=+1528.955762443"
Aug 04 11:58:41.172008 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:41.171964    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:58:44.518121 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:44.518084    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:44.518640 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:44.518146    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:44.530713 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:44.530679    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:45.427556 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:45.427529    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:58:55.836615 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:58:55.836572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:59:31.736734 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:31.736700    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"]
Aug 04 11:59:31.756567 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:31.756538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:59:33.253875 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:33.253802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 11:59:46.757563 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:46.757500    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472" podUID="4c2b3b37-7cfd-46f1-99c9-65753d9805fc" containerName="main" containerID="cri-o://67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1" gracePeriod=45
Aug 04 11:59:47.002892 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.002861    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:59:47.088334 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088229    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-model-cache\") pod \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") "
Aug 04 11:59:47.088334 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088264    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tmp-dir\") pod \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") "
Aug 04 11:59:47.088334 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088324    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrzrs\" (UniqueName: \"kubernetes.io/projected/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-kube-api-access-qrzrs\") pod \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") "
Aug 04 11:59:47.088617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088397    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tls-certs\") pod \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") "
Aug 04 11:59:47.088617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-dshm\") pod \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") "
Aug 04 11:59:47.088617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088440    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-home\") pod \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\" (UID: \"4c2b3b37-7cfd-46f1-99c9-65753d9805fc\") "
Aug 04 11:59:47.088617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088551    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c2b3b37-7cfd-46f1-99c9-65753d9805fc" (UID: "4c2b3b37-7cfd-46f1-99c9-65753d9805fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:59:47.088617 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088592    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c2b3b37-7cfd-46f1-99c9-65753d9805fc" (UID: "4c2b3b37-7cfd-46f1-99c9-65753d9805fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:59:47.088808 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088712    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:59:47.088808 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088754    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:59:47.088913 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.088829    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-home" (OuterVolumeSpecName: "home") pod "4c2b3b37-7cfd-46f1-99c9-65753d9805fc" (UID: "4c2b3b37-7cfd-46f1-99c9-65753d9805fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:59:47.090696 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.090672    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-dshm" (OuterVolumeSpecName: "dshm") pod "4c2b3b37-7cfd-46f1-99c9-65753d9805fc" (UID: "4c2b3b37-7cfd-46f1-99c9-65753d9805fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 11:59:47.090779 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.090718    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c2b3b37-7cfd-46f1-99c9-65753d9805fc" (UID: "4c2b3b37-7cfd-46f1-99c9-65753d9805fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 11:59:47.090818 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.090796    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-kube-api-access-qrzrs" (OuterVolumeSpecName: "kube-api-access-qrzrs") pod "4c2b3b37-7cfd-46f1-99c9-65753d9805fc" (UID: "4c2b3b37-7cfd-46f1-99c9-65753d9805fc"). InnerVolumeSpecName "kube-api-access-qrzrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 11:59:47.190180 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.190145    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:59:47.190180 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.190174    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:59:47.190180 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.190186    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:59:47.190438 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.190197    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrzrs\" (UniqueName: \"kubernetes.io/projected/4c2b3b37-7cfd-46f1-99c9-65753d9805fc-kube-api-access-qrzrs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 11:59:47.731247 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.731215    2558 generic.go:356] "Generic (PLEG): container finished" podID="4c2b3b37-7cfd-46f1-99c9-65753d9805fc" containerID="67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1" exitCode=0
Aug 04 11:59:47.731440 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.731258    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"
Aug 04 11:59:47.731440 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.731307    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472" event={"ID":"4c2b3b37-7cfd-46f1-99c9-65753d9805fc","Type":"ContainerDied","Data":"67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1"}
Aug 04 11:59:47.731440 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.731354    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472" event={"ID":"4c2b3b37-7cfd-46f1-99c9-65753d9805fc","Type":"ContainerDied","Data":"001b641200ed007759d6a9800013c0c87aaadfc10f1858aabf6e92e9168f9438"}
Aug 04 11:59:47.731440 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.731372    2558 scope.go:117] "RemoveContainer" containerID="67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1"
Aug 04 11:59:47.740439 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.740423    2558 scope.go:117] "RemoveContainer" containerID="67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1"
Aug 04 11:59:47.740681 ip-10-0-129-11 kubenswrapper[2558]: E0804 11:59:47.740664    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1\": container with ID starting with 67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1 not found: ID does not exist" containerID="67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1"
Aug 04 11:59:47.740720 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.740689    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1"} err="failed to get container status \"67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1\": rpc error: code = NotFound desc = could not find container \"67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1\": container with ID starting with 67902597fb67e74465e0b1e63bdd08adb61af2b06b959d8ff844f54238dbefa1 not found: ID does not exist"
Aug 04 11:59:47.753654 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.753634    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"]
Aug 04 11:59:47.759287 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:47.759259    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d25909fe-kserve-5dbccdd687-v2472"]
Aug 04 11:59:48.994145 ip-10-0-129-11 kubenswrapper[2558]: I0804 11:59:48.994108    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2b3b37-7cfd-46f1-99c9-65753d9805fc" path="/var/lib/kubelet/pods/4c2b3b37-7cfd-46f1-99c9-65753d9805fc/volumes"
Aug 04 12:00:00.132962 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.132879    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"]
Aug 04 12:00:00.133376 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.133260    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2b3b37-7cfd-46f1-99c9-65753d9805fc" containerName="main"
Aug 04 12:00:00.133376 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.133271    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2b3b37-7cfd-46f1-99c9-65753d9805fc" containerName="main"
Aug 04 12:00:00.133376 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.133375    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2b3b37-7cfd-46f1-99c9-65753d9805fc" containerName="main"
Aug 04 12:00:00.137694 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.137674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"
Aug 04 12:00:00.140647 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.140621    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:00:00.145418 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.145375    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"]
Aug 04 12:00:00.155586 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.155552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"
Aug 04 12:00:00.297955 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.297782    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"]
Aug 04 12:00:00.300692 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:00:00.300665    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2986b88_8d68_49a5_8d31_f32bccb7000d.slice/crio-61c462c23414bc99cf12d3437b7274d55c291e1ee9000be18fa8e99c3186fdaf WatchSource:0}: Error finding container 61c462c23414bc99cf12d3437b7274d55c291e1ee9000be18fa8e99c3186fdaf: Status 404 returned error can't find the container with id 61c462c23414bc99cf12d3437b7274d55c291e1ee9000be18fa8e99c3186fdaf
Aug 04 12:00:00.797624 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.797585    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf" event={"ID":"e2986b88-8d68-49a5-8d31-f32bccb7000d","Type":"ContainerStarted","Data":"ddaf6dac7ca335415daa95f8c8b078315258c99031aa503a1ee8650209ae64cf"}
Aug 04 12:00:00.797814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:00.797650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf" event={"ID":"e2986b88-8d68-49a5-8d31-f32bccb7000d","Type":"ContainerStarted","Data":"61c462c23414bc99cf12d3437b7274d55c291e1ee9000be18fa8e99c3186fdaf"}
Aug 04 12:00:01.803814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:01.803777    2558 generic.go:356] "Generic (PLEG): container finished" podID="e2986b88-8d68-49a5-8d31-f32bccb7000d" containerID="ddaf6dac7ca335415daa95f8c8b078315258c99031aa503a1ee8650209ae64cf" exitCode=0
Aug 04 12:00:01.804268 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:01.803859    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf" event={"ID":"e2986b88-8d68-49a5-8d31-f32bccb7000d","Type":"ContainerDied","Data":"ddaf6dac7ca335415daa95f8c8b078315258c99031aa503a1ee8650209ae64cf"}
Aug 04 12:00:02.939744 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:02.939721    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"
Aug 04 12:00:03.814672 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:03.814642    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf"
Aug 04 12:00:03.814843 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:03.814641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764080-jtllf" event={"ID":"e2986b88-8d68-49a5-8d31-f32bccb7000d","Type":"ContainerDied","Data":"61c462c23414bc99cf12d3437b7274d55c291e1ee9000be18fa8e99c3186fdaf"}
Aug 04 12:00:03.814843 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:03.814769    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61c462c23414bc99cf12d3437b7274d55c291e1ee9000be18fa8e99c3186fdaf"
Aug 04 12:00:30.992141 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:30.992106    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:00:33.616945 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:00:33.616908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:05.061274 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:05.061239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:07.015389 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:07.015359    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:03:07.022033 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:07.022008    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:03:30.887549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:30.887519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:33.829857 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:33.829822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:33.926261 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:33.926230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:34.256035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:34.256005    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:40.251648 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:40.251616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:40.352009 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:40.351980    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:40.478243 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:40.478215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:40.781663 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:40.781626    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:40.973912 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:40.973875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:42.799387 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:42.799354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:42.882781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:42.882748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:43.869078 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.869036    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"]
Aug 04 12:03:43.869666 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.869645    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2986b88-8d68-49a5-8d31-f32bccb7000d" containerName="cleanup"
Aug 04 12:03:43.869711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.869671    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2986b88-8d68-49a5-8d31-f32bccb7000d" containerName="cleanup"
Aug 04 12:03:43.869790 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.869778    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2986b88-8d68-49a5-8d31-f32bccb7000d" containerName="cleanup"
Aug 04 12:03:43.872913 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.872896    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:43.875865 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.875836    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7db38b79-serving-cert\""
Aug 04 12:03:43.883671 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.883648    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"]
Aug 04 12:03:43.903055 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.903029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:43.975625 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.975591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33abc94a-92cf-4655-891a-f1a4e557494b-openshift-service-ca\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:43.975796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.975634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4wdr\" (UniqueName: \"kubernetes.io/projected/33abc94a-92cf-4655-891a-f1a4e557494b-kube-api-access-f4wdr\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:43.975796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:43.975680    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.076306 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.076262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33abc94a-92cf-4655-891a-f1a4e557494b-openshift-service-ca\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.076494 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.076324    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4wdr\" (UniqueName: \"kubernetes.io/projected/33abc94a-92cf-4655-891a-f1a4e557494b-kube-api-access-f4wdr\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.076494 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.076383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.076618 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:03:44.076545    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-serving-cert: secret "maas-api-e2e-ait-7db38b79-serving-cert" not found
Aug 04 12:03:44.076679 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:03:44.076638    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls podName:33abc94a-92cf-4655-891a-f1a4e557494b nodeName:}" failed. No retries permitted until 2026-08-04 12:03:44.576613996 +0000 UTC m=+1838.147526792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls") pod "maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" (UID: "33abc94a-92cf-4655-891a-f1a4e557494b") : secret "maas-api-e2e-ait-7db38b79-serving-cert" not found
Aug 04 12:03:44.076967 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.076946    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33abc94a-92cf-4655-891a-f1a4e557494b-openshift-service-ca\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.089105 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.089085    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4wdr\" (UniqueName: \"kubernetes.io/projected/33abc94a-92cf-4655-891a-f1a4e557494b-kube-api-access-f4wdr\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.484450 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.484412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:44.576504 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.576464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:44.580688 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.580659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:44.580818 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:03:44.580767    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-serving-cert: secret "maas-api-e2e-ait-7db38b79-serving-cert" not found
Aug 04 12:03:44.580818 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:03:44.580818    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls podName:33abc94a-92cf-4655-891a-f1a4e557494b nodeName:}" failed. No retries permitted until 2026-08-04 12:03:45.580805169 +0000 UTC m=+1839.151717950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls") pod "maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" (UID: "33abc94a-92cf-4655-891a-f1a4e557494b") : secret "maas-api-e2e-ait-7db38b79-serving-cert" not found
Aug 04 12:03:44.603537 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.603510    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"]
Aug 04 12:03:44.607241 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.607219    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.609854 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.609833    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7db38b79\""
Aug 04 12:03:44.609955 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.609907    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7db38b79-dockercfg-gk2n6\""
Aug 04 12:03:44.619041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.619011    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"]
Aug 04 12:03:44.782547 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.782518    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx86l\" (UniqueName: \"kubernetes.io/projected/953f774f-ce60-4793-a232-20d724a48db4-kube-api-access-lx86l\") pod \"payload-processing-e2e-ait-7db38b79-97f947bd-brpq6\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") " pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.782710 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.782661    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953f774f-ce60-4793-a232-20d724a48db4-plugins-config-volume\") pod \"payload-processing-e2e-ait-7db38b79-97f947bd-brpq6\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") " pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.883121 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.883084    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953f774f-ce60-4793-a232-20d724a48db4-plugins-config-volume\") pod \"payload-processing-e2e-ait-7db38b79-97f947bd-brpq6\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") " pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.883571 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.883148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx86l\" (UniqueName: \"kubernetes.io/projected/953f774f-ce60-4793-a232-20d724a48db4-kube-api-access-lx86l\") pod \"payload-processing-e2e-ait-7db38b79-97f947bd-brpq6\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") " pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.883899 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.883872    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953f774f-ce60-4793-a232-20d724a48db4-plugins-config-volume\") pod \"payload-processing-e2e-ait-7db38b79-97f947bd-brpq6\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") " pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.892749 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.892723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx86l\" (UniqueName: \"kubernetes.io/projected/953f774f-ce60-4793-a232-20d724a48db4-kube-api-access-lx86l\") pod \"payload-processing-e2e-ait-7db38b79-97f947bd-brpq6\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") " pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:44.893374 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.893342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:03:44.920034 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:44.920015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:45.065756 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:45.065728    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"]
Aug 04 12:03:45.068052 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:03:45.068028    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953f774f_ce60_4793_a232_20d724a48db4.slice/crio-9ef6bbc9e7e67e0cfe1447243538f951b2d61a20c38f6daa5a81fc7466ffe85e WatchSource:0}: Error finding container 9ef6bbc9e7e67e0cfe1447243538f951b2d61a20c38f6daa5a81fc7466ffe85e: Status 404 returned error can't find the container with id 9ef6bbc9e7e67e0cfe1447243538f951b2d61a20c38f6daa5a81fc7466ffe85e
Aug 04 12:03:45.070084 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:45.070065    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 12:03:45.589298 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:45.589242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:45.591704 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:45.591679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") pod \"maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:45.684722 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:45.684685    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:45.916501 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:45.916413    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6" event={"ID":"953f774f-ce60-4793-a232-20d724a48db4","Type":"ContainerStarted","Data":"9ef6bbc9e7e67e0cfe1447243538f951b2d61a20c38f6daa5a81fc7466ffe85e"}
Aug 04 12:03:46.030073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:46.030041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"]
Aug 04 12:03:46.033392 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:03:46.033362    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33abc94a_92cf_4655_891a_f1a4e557494b.slice/crio-b87c42e64ac1d3b2ef9bb803d8d2f3a927f62975299c5857b00c4c5c1a874ab6 WatchSource:0}: Error finding container b87c42e64ac1d3b2ef9bb803d8d2f3a927f62975299c5857b00c4c5c1a874ab6: Status 404 returned error can't find the container with id b87c42e64ac1d3b2ef9bb803d8d2f3a927f62975299c5857b00c4c5c1a874ab6
Aug 04 12:03:46.922720 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:46.922692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" event={"ID":"33abc94a-92cf-4655-891a-f1a4e557494b","Type":"ContainerStarted","Data":"9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9"}
Aug 04 12:03:46.923195 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:46.922741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" event={"ID":"33abc94a-92cf-4655-891a-f1a4e557494b","Type":"ContainerStarted","Data":"b87c42e64ac1d3b2ef9bb803d8d2f3a927f62975299c5857b00c4c5c1a874ab6"}
Aug 04 12:03:46.923195 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:46.922826    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:46.941790 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:46.941734    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" podStartSLOduration=3.163816274 podStartE2EDuration="3.941712886s" totalImagesPullingTime="777.896612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:03:46.939129169 +0000 UTC m=+1840.510042066" watchObservedRunningTime="2026-08-04 12:03:46.941712886 +0000 UTC m=+1840.512625693"
Aug 04 12:03:47.929000 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:47.928964    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6" event={"ID":"953f774f-ce60-4793-a232-20d724a48db4","Type":"ContainerStarted","Data":"aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740"}
Aug 04 12:03:47.947940 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:47.947897    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6" podStartSLOduration=2.151277271 podStartE2EDuration="3.947883199s" totalImagesPullingTime="1.796605928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:03:47.945816798 +0000 UTC m=+1841.516729604" watchObservedRunningTime="2026-08-04 12:03:47.947883199 +0000 UTC m=+1841.518796029"
Aug 04 12:03:48.933508 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:48.933471    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:03:52.935216 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:52.935175    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:03:59.940417 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:03:59.940383    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:04:08.328266 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.328222    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"]
Aug 04 12:04:08.334985 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.334956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:08.337676 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.337646    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"]
Aug 04 12:04:08.337867 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.337845    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:04:08.365099 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.365068    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b16d589-5a3d-4a66-8819-08b6755c8b14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7db38b79-242fw\" (UID: \"9b16d589-5a3d-4a66-8819-08b6755c8b14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:08.466455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.466420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b16d589-5a3d-4a66-8819-08b6755c8b14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7db38b79-242fw\" (UID: \"9b16d589-5a3d-4a66-8819-08b6755c8b14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:08.467147 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.467129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b16d589-5a3d-4a66-8819-08b6755c8b14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7db38b79-242fw\" (UID: \"9b16d589-5a3d-4a66-8819-08b6755c8b14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:08.646041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.645950    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:08.795382 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:08.795351    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"]
Aug 04 12:04:08.797809 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:04:08.797789    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b16d589_5a3d_4a66_8819_08b6755c8b14.slice/crio-8ff7c7c58b538671c80ff143bfa8898695a1ba32931267c6ac478408387527d6 WatchSource:0}: Error finding container 8ff7c7c58b538671c80ff143bfa8898695a1ba32931267c6ac478408387527d6: Status 404 returned error can't find the container with id 8ff7c7c58b538671c80ff143bfa8898695a1ba32931267c6ac478408387527d6
Aug 04 12:04:09.033131 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:09.033098    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw" event={"ID":"9b16d589-5a3d-4a66-8819-08b6755c8b14","Type":"ContainerStarted","Data":"8ff7c7c58b538671c80ff143bfa8898695a1ba32931267c6ac478408387527d6"}
Aug 04 12:04:10.039293 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:10.039252    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b16d589-5a3d-4a66-8819-08b6755c8b14" containerID="31c4c15518360f43e2939f30bfa68a421c13bb50e28f35f835a268d0b9cfbbba" exitCode=0
Aug 04 12:04:10.039669 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:10.039344    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw" event={"ID":"9b16d589-5a3d-4a66-8819-08b6755c8b14","Type":"ContainerDied","Data":"31c4c15518360f43e2939f30bfa68a421c13bb50e28f35f835a268d0b9cfbbba"}
Aug 04 12:04:11.169857 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:11.169834    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:11.187027 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:11.187001    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b16d589-5a3d-4a66-8819-08b6755c8b14-maas-api-service-ca\") pod \"9b16d589-5a3d-4a66-8819-08b6755c8b14\" (UID: \"9b16d589-5a3d-4a66-8819-08b6755c8b14\") "
Aug 04 12:04:11.187419 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:11.187359    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b16d589-5a3d-4a66-8819-08b6755c8b14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b16d589-5a3d-4a66-8819-08b6755c8b14" (UID: "9b16d589-5a3d-4a66-8819-08b6755c8b14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:04:11.288034 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:11.288005    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b16d589-5a3d-4a66-8819-08b6755c8b14-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:12.051035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:12.051002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw" event={"ID":"9b16d589-5a3d-4a66-8819-08b6755c8b14","Type":"ContainerDied","Data":"8ff7c7c58b538671c80ff143bfa8898695a1ba32931267c6ac478408387527d6"}
Aug 04 12:04:12.051035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:12.051039    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"
Aug 04 12:04:12.051248 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:12.051048    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ff7c7c58b538671c80ff143bfa8898695a1ba32931267c6ac478408387527d6"
Aug 04 12:04:14.321728 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.321690    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"]
Aug 04 12:04:14.322105 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.321940    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" podUID="33abc94a-92cf-4655-891a-f1a4e557494b" containerName="maas-api" containerID="cri-o://9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9" gracePeriod=30
Aug 04 12:04:14.347421 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.347396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:14.372895 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.372863    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"]
Aug 04 12:04:14.373089 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.373068    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6" podUID="953f774f-ce60-4793-a232-20d724a48db4" containerName="payload-processing" containerID="cri-o://aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740" gracePeriod=30
Aug 04 12:04:14.447606 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.447574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:14.525294 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.525207    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"]
Aug 04 12:04:14.534955 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.533353    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7db38b79-242fw"]
Aug 04 12:04:14.535947 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.535915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:14.613264 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.613233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:14.656798 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.656705    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:04:14.660573 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.660550    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:04:14.688649 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.688620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:14.713868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.713841    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33abc94a-92cf-4655-891a-f1a4e557494b-openshift-service-ca\") pod \"33abc94a-92cf-4655-891a-f1a4e557494b\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") "
Aug 04 12:04:14.714043 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.713873    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") pod \"33abc94a-92cf-4655-891a-f1a4e557494b\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") "
Aug 04 12:04:14.714043 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.713893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4wdr\" (UniqueName: \"kubernetes.io/projected/33abc94a-92cf-4655-891a-f1a4e557494b-kube-api-access-f4wdr\") pod \"33abc94a-92cf-4655-891a-f1a4e557494b\" (UID: \"33abc94a-92cf-4655-891a-f1a4e557494b\") "
Aug 04 12:04:14.714165 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.714098    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx86l\" (UniqueName: \"kubernetes.io/projected/953f774f-ce60-4793-a232-20d724a48db4-kube-api-access-lx86l\") pod \"953f774f-ce60-4793-a232-20d724a48db4\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") "
Aug 04 12:04:14.714223 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.714214    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953f774f-ce60-4793-a232-20d724a48db4-plugins-config-volume\") pod \"953f774f-ce60-4793-a232-20d724a48db4\" (UID: \"953f774f-ce60-4793-a232-20d724a48db4\") "
Aug 04 12:04:14.714310 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.714242    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33abc94a-92cf-4655-891a-f1a4e557494b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33abc94a-92cf-4655-891a-f1a4e557494b" (UID: "33abc94a-92cf-4655-891a-f1a4e557494b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:04:14.714563 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.714528    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33abc94a-92cf-4655-891a-f1a4e557494b-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:14.714763 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.714736    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953f774f-ce60-4793-a232-20d724a48db4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "953f774f-ce60-4793-a232-20d724a48db4" (UID: "953f774f-ce60-4793-a232-20d724a48db4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:04:14.716201 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.716180    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33abc94a-92cf-4655-891a-f1a4e557494b-kube-api-access-f4wdr" (OuterVolumeSpecName: "kube-api-access-f4wdr") pod "33abc94a-92cf-4655-891a-f1a4e557494b" (UID: "33abc94a-92cf-4655-891a-f1a4e557494b"). InnerVolumeSpecName "kube-api-access-f4wdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:04:14.716709 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.716687    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33abc94a-92cf-4655-891a-f1a4e557494b" (UID: "33abc94a-92cf-4655-891a-f1a4e557494b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:04:14.716774 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.716690    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953f774f-ce60-4793-a232-20d724a48db4-kube-api-access-lx86l" (OuterVolumeSpecName: "kube-api-access-lx86l") pod "953f774f-ce60-4793-a232-20d724a48db4" (UID: "953f774f-ce60-4793-a232-20d724a48db4"). InnerVolumeSpecName "kube-api-access-lx86l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:04:14.815206 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.815173    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953f774f-ce60-4793-a232-20d724a48db4-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:14.815206 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.815201    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33abc94a-92cf-4655-891a-f1a4e557494b-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:14.815436 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.815214    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4wdr\" (UniqueName: \"kubernetes.io/projected/33abc94a-92cf-4655-891a-f1a4e557494b-kube-api-access-f4wdr\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:14.815436 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.815227    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx86l\" (UniqueName: \"kubernetes.io/projected/953f774f-ce60-4793-a232-20d724a48db4-kube-api-access-lx86l\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:14.910026 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.909941    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:14.994042 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:14.994000    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b16d589-5a3d-4a66-8819-08b6755c8b14" path="/var/lib/kubelet/pods/9b16d589-5a3d-4a66-8819-08b6755c8b14/volumes"
Aug 04 12:04:15.066746 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.066715    2558 generic.go:356] "Generic (PLEG): container finished" podID="33abc94a-92cf-4655-891a-f1a4e557494b" containerID="9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9" exitCode=0
Aug 04 12:04:15.066906 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.066766    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"
Aug 04 12:04:15.066906 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.066772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" event={"ID":"33abc94a-92cf-4655-891a-f1a4e557494b","Type":"ContainerDied","Data":"9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9"}
Aug 04 12:04:15.066906 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.066824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb" event={"ID":"33abc94a-92cf-4655-891a-f1a4e557494b","Type":"ContainerDied","Data":"b87c42e64ac1d3b2ef9bb803d8d2f3a927f62975299c5857b00c4c5c1a874ab6"}
Aug 04 12:04:15.066906 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.066846    2558 scope.go:117] "RemoveContainer" containerID="9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9"
Aug 04 12:04:15.068129 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.068101    2558 generic.go:356] "Generic (PLEG): container finished" podID="953f774f-ce60-4793-a232-20d724a48db4" containerID="aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740" exitCode=0
Aug 04 12:04:15.068239 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.068144    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"
Aug 04 12:04:15.068239 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.068199    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6" event={"ID":"953f774f-ce60-4793-a232-20d724a48db4","Type":"ContainerDied","Data":"aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740"}
Aug 04 12:04:15.068371 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.068240    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6" event={"ID":"953f774f-ce60-4793-a232-20d724a48db4","Type":"ContainerDied","Data":"9ef6bbc9e7e67e0cfe1447243538f951b2d61a20c38f6daa5a81fc7466ffe85e"}
Aug 04 12:04:15.077044 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.077028    2558 scope.go:117] "RemoveContainer" containerID="9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9"
Aug 04 12:04:15.077313 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:04:15.077290    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9\": container with ID starting with 9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9 not found: ID does not exist" containerID="9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9"
Aug 04 12:04:15.077369 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.077321    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9"} err="failed to get container status \"9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9\": rpc error: code = NotFound desc = could not find container \"9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9\": container with ID starting with 9cf730840ac3b02984f334f29aa9dda899d3fdd3af46b8ad0c6fd521a12f8ba9 not found: ID does not exist"
Aug 04 12:04:15.077369 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.077336    2558 scope.go:117] "RemoveContainer" containerID="aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740"
Aug 04 12:04:15.085407 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.085310    2558 scope.go:117] "RemoveContainer" containerID="aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740"
Aug 04 12:04:15.085599 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:04:15.085578    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740\": container with ID starting with aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740 not found: ID does not exist" containerID="aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740"
Aug 04 12:04:15.085665 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.085604    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740"} err="failed to get container status \"aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740\": rpc error: code = NotFound desc = could not find container \"aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740\": container with ID starting with aa24e7abb0258baa00417755456c51279f6bcc81bdbc03cf4c8e75ba3a4f2740 not found: ID does not exist"
Aug 04 12:04:15.086776 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.086756    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"]
Aug 04 12:04:15.092425 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.092404    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6"]
Aug 04 12:04:15.109896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.109876    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"]
Aug 04 12:04:15.117993 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.117974    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7db38b79-54b54d7b9-6qffb"]
Aug 04 12:04:15.166632 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.166555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:15.284511 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.284470    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:15.379519 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.379488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:15.714663 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.714629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:15.834223 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:15.834192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:16.994039 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:16.994000    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33abc94a-92cf-4655-891a-f1a4e557494b" path="/var/lib/kubelet/pods/33abc94a-92cf-4655-891a-f1a4e557494b/volumes"
Aug 04 12:04:16.994574 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:16.994552    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953f774f-ce60-4793-a232-20d724a48db4" path="/var/lib/kubelet/pods/953f774f-ce60-4793-a232-20d724a48db4/volumes"
Aug 04 12:04:18.091985 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.091954    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:18.183385 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183339    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"]
Aug 04 12:04:18.183711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183696    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953f774f-ce60-4793-a232-20d724a48db4" containerName="payload-processing"
Aug 04 12:04:18.183755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183712    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="953f774f-ce60-4793-a232-20d724a48db4" containerName="payload-processing"
Aug 04 12:04:18.183755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183722    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33abc94a-92cf-4655-891a-f1a4e557494b" containerName="maas-api"
Aug 04 12:04:18.183755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183728    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33abc94a-92cf-4655-891a-f1a4e557494b" containerName="maas-api"
Aug 04 12:04:18.183755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183745    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b16d589-5a3d-4a66-8819-08b6755c8b14" containerName="revoke-keys"
Aug 04 12:04:18.183755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183751    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b16d589-5a3d-4a66-8819-08b6755c8b14" containerName="revoke-keys"
Aug 04 12:04:18.183911 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183805    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="953f774f-ce60-4793-a232-20d724a48db4" containerName="payload-processing"
Aug 04 12:04:18.183911 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183816    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b16d589-5a3d-4a66-8819-08b6755c8b14" containerName="revoke-keys"
Aug 04 12:04:18.183911 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.183824    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="33abc94a-92cf-4655-891a-f1a4e557494b" containerName="maas-api"
Aug 04 12:04:18.188100 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.188083    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.191050 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.191032    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b6e21d41-serving-cert\""
Aug 04 12:04:18.197379 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.197350    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"]
Aug 04 12:04:18.245577 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.245542    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-openshift-service-ca\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.245741 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.245647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89z4j\" (UniqueName: \"kubernetes.io/projected/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-kube-api-access-89z4j\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.245817 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.245796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.270858 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.270831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:18.347054 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.346978    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.347054 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.347024    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-openshift-service-ca\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.347237 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:04:18.347129    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-serving-cert: secret "maas-api-e2e-migrate-b6e21d41-serving-cert" not found
Aug 04 12:04:18.347237 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.347149    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89z4j\" (UniqueName: \"kubernetes.io/projected/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-kube-api-access-89z4j\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.347237 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:04:18.347212    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls podName:82e39b82-6dfa-4daf-ad19-b31ff6900d9e nodeName:}" failed. No retries permitted until 2026-08-04 12:04:18.847190475 +0000 UTC m=+1872.418103259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls") pod "maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" (UID: "82e39b82-6dfa-4daf-ad19-b31ff6900d9e") : secret "maas-api-e2e-migrate-b6e21d41-serving-cert" not found
Aug 04 12:04:18.347711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.347692    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-openshift-service-ca\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.357472 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.357445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89z4j\" (UniqueName: \"kubernetes.io/projected/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-kube-api-access-89z4j\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.536100 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.536068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:18.739513 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.739418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:18.840786 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.840753    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:18.852777 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.852747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.855136 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.855112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls\") pod \"maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:18.880579 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.880553    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"]
Aug 04 12:04:18.883949 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.883929    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:18.886858 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.886840    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b6e21d41-dockercfg-7dhwh\""
Aug 04 12:04:18.887088 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.887073    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b6e21d41\""
Aug 04 12:04:18.896556 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.896530    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"]
Aug 04 12:04:18.953643 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.953603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:18.953816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:18.953679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdb6q\" (UniqueName: \"kubernetes.io/projected/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-kube-api-access-fdb6q\") pod \"payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:19.054530 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.054487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdb6q\" (UniqueName: \"kubernetes.io/projected/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-kube-api-access-fdb6q\") pod \"payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:19.054713 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.054651    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:19.055331 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.055312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:19.065105 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.065079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdb6q\" (UniqueName: \"kubernetes.io/projected/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-kube-api-access-fdb6q\") pod \"payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:19.099563 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.099535    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:19.197682 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.197644    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:19.349565 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.349484    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"]
Aug 04 12:04:19.352329 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:04:19.352295    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cbaf32_883a_4fd3_a8d4_b2a0743f2af9.slice/crio-b1961a2e5ab9dabd9522d0f5728651e12922bce6c30060bcd41f4ffd945670d5 WatchSource:0}: Error finding container b1961a2e5ab9dabd9522d0f5728651e12922bce6c30060bcd41f4ffd945670d5: Status 404 returned error can't find the container with id b1961a2e5ab9dabd9522d0f5728651e12922bce6c30060bcd41f4ffd945670d5
Aug 04 12:04:19.445475 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:19.445426    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"]
Aug 04 12:04:19.447762 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:04:19.447732    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e39b82_6dfa_4daf_ad19_b31ff6900d9e.slice/crio-f4ba5a694d2898c9ce1d67801a2d6d0777a0853829686408fd67567fb0c0b4b3 WatchSource:0}: Error finding container f4ba5a694d2898c9ce1d67801a2d6d0777a0853829686408fd67567fb0c0b4b3: Status 404 returned error can't find the container with id f4ba5a694d2898c9ce1d67801a2d6d0777a0853829686408fd67567fb0c0b4b3
Aug 04 12:04:20.097124 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.097088    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" event={"ID":"82e39b82-6dfa-4daf-ad19-b31ff6900d9e","Type":"ContainerStarted","Data":"7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb"}
Aug 04 12:04:20.097332 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.097149    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:20.097332 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.097175    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" event={"ID":"82e39b82-6dfa-4daf-ad19-b31ff6900d9e","Type":"ContainerStarted","Data":"f4ba5a694d2898c9ce1d67801a2d6d0777a0853829686408fd67567fb0c0b4b3"}
Aug 04 12:04:20.098548 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.098522    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz" event={"ID":"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9","Type":"ContainerStarted","Data":"b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8"}
Aug 04 12:04:20.098667 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.098572    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:20.098667 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.098594    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz" event={"ID":"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9","Type":"ContainerStarted","Data":"b1961a2e5ab9dabd9522d0f5728651e12922bce6c30060bcd41f4ffd945670d5"}
Aug 04 12:04:20.118374 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.118287    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" podStartSLOduration=1.699765744 podStartE2EDuration="2.118262581s" totalImagesPullingTime="418.496837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:04:20.116387199 +0000 UTC m=+1873.687300016" watchObservedRunningTime="2026-08-04 12:04:20.118262581 +0000 UTC m=+1873.689175384"
Aug 04 12:04:20.134391 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:20.134349    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz" podStartSLOduration=2.134336254 podStartE2EDuration="2.134336254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:04:20.132511321 +0000 UTC m=+1873.703424126" watchObservedRunningTime="2026-08-04 12:04:20.134336254 +0000 UTC m=+1873.705249057"
Aug 04 12:04:26.110299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:26.110249    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:27.318184 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:27.318149    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:31.106342 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:31.106249    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:43.061929 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.061900    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"]
Aug 04 12:04:43.065531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.065507    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:43.068503 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.068481    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:04:43.075064 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.072984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"]
Aug 04 12:04:43.164733 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.164695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28bf982f-967f-4431-81e0-11346178fb3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr\" (UID: \"28bf982f-967f-4431-81e0-11346178fb3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:43.266191 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.266158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28bf982f-967f-4431-81e0-11346178fb3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr\" (UID: \"28bf982f-967f-4431-81e0-11346178fb3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:43.266794 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.266774    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28bf982f-967f-4431-81e0-11346178fb3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr\" (UID: \"28bf982f-967f-4431-81e0-11346178fb3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:43.379307 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.379216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:43.514392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:43.514365    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"]
Aug 04 12:04:43.516066 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:04:43.516039    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28bf982f_967f_4431_81e0_11346178fb3f.slice/crio-a38ffb79524304b789cf79e4ecf7204f8ae0a37951beba68fc8ee053ea950a22 WatchSource:0}: Error finding container a38ffb79524304b789cf79e4ecf7204f8ae0a37951beba68fc8ee053ea950a22: Status 404 returned error can't find the container with id a38ffb79524304b789cf79e4ecf7204f8ae0a37951beba68fc8ee053ea950a22
Aug 04 12:04:44.226662 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:44.226635    2558 generic.go:356] "Generic (PLEG): container finished" podID="28bf982f-967f-4431-81e0-11346178fb3f" containerID="731c8b9257510f6e2c56b023cb0212c76cf21df04c3a740501ab759fc71d9459" exitCode=0
Aug 04 12:04:44.226980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:44.226714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr" event={"ID":"28bf982f-967f-4431-81e0-11346178fb3f","Type":"ContainerDied","Data":"731c8b9257510f6e2c56b023cb0212c76cf21df04c3a740501ab759fc71d9459"}
Aug 04 12:04:44.226980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:44.226764    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr" event={"ID":"28bf982f-967f-4431-81e0-11346178fb3f","Type":"ContainerStarted","Data":"a38ffb79524304b789cf79e4ecf7204f8ae0a37951beba68fc8ee053ea950a22"}
Aug 04 12:04:45.383598 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:45.383575    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:45.486566 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:45.486494    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28bf982f-967f-4431-81e0-11346178fb3f-maas-api-service-ca\") pod \"28bf982f-967f-4431-81e0-11346178fb3f\" (UID: \"28bf982f-967f-4431-81e0-11346178fb3f\") "
Aug 04 12:04:45.486826 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:45.486805    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bf982f-967f-4431-81e0-11346178fb3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28bf982f-967f-4431-81e0-11346178fb3f" (UID: "28bf982f-967f-4431-81e0-11346178fb3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:04:45.588063 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:45.588033    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28bf982f-967f-4431-81e0-11346178fb3f-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:46.237549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:46.237517    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr" event={"ID":"28bf982f-967f-4431-81e0-11346178fb3f","Type":"ContainerDied","Data":"a38ffb79524304b789cf79e4ecf7204f8ae0a37951beba68fc8ee053ea950a22"}
Aug 04 12:04:46.237549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:46.237540    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"
Aug 04 12:04:46.237764 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:46.237571    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a38ffb79524304b789cf79e4ecf7204f8ae0a37951beba68fc8ee053ea950a22"
Aug 04 12:04:48.812566 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:48.812532    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"]
Aug 04 12:04:48.812975 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:48.812743    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" podUID="82e39b82-6dfa-4daf-ad19-b31ff6900d9e" containerName="maas-api" containerID="cri-o://7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb" gracePeriod=30
Aug 04 12:04:48.829607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:48.829582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:48.864930 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:48.864899    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"]
Aug 04 12:04:48.865166 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:48.865141    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz" podUID="b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" containerName="payload-processing" containerID="cri-o://b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8" gracePeriod=30
Aug 04 12:04:48.927939 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:48.927909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:49.005540 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.004953    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"]
Aug 04 12:04:49.008916 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.008790    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6e21d41-6x9vr"]
Aug 04 12:04:49.009087 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.009012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:49.074572 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.074538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:49.122353 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.122328    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:49.156247 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.156221    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:49.167456 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.167427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:49.220685 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.220657    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls\") pod \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") "
Aug 04 12:04:49.220883 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.220722    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-openshift-service-ca\") pod \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") "
Aug 04 12:04:49.220883 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.220751    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89z4j\" (UniqueName: \"kubernetes.io/projected/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-kube-api-access-89z4j\") pod \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\" (UID: \"82e39b82-6dfa-4daf-ad19-b31ff6900d9e\") "
Aug 04 12:04:49.221143 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.221123    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82e39b82-6dfa-4daf-ad19-b31ff6900d9e" (UID: "82e39b82-6dfa-4daf-ad19-b31ff6900d9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:04:49.222853 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.222832    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-kube-api-access-89z4j" (OuterVolumeSpecName: "kube-api-access-89z4j") pod "82e39b82-6dfa-4daf-ad19-b31ff6900d9e" (UID: "82e39b82-6dfa-4daf-ad19-b31ff6900d9e"). InnerVolumeSpecName "kube-api-access-89z4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:04:49.222940 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.222872    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82e39b82-6dfa-4daf-ad19-b31ff6900d9e" (UID: "82e39b82-6dfa-4daf-ad19-b31ff6900d9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:04:49.254397 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.254369    2558 generic.go:356] "Generic (PLEG): container finished" podID="b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" containerID="b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8" exitCode=0
Aug 04 12:04:49.254549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.254410    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"
Aug 04 12:04:49.254549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.254458    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz" event={"ID":"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9","Type":"ContainerDied","Data":"b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8"}
Aug 04 12:04:49.254549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.254505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz" event={"ID":"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9","Type":"ContainerDied","Data":"b1961a2e5ab9dabd9522d0f5728651e12922bce6c30060bcd41f4ffd945670d5"}
Aug 04 12:04:49.254549 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.254526    2558 scope.go:117] "RemoveContainer" containerID="b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8"
Aug 04 12:04:49.255780 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.255758    2558 generic.go:356] "Generic (PLEG): container finished" podID="82e39b82-6dfa-4daf-ad19-b31ff6900d9e" containerID="7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb" exitCode=0
Aug 04 12:04:49.255896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.255800    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"
Aug 04 12:04:49.255896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.255846    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" event={"ID":"82e39b82-6dfa-4daf-ad19-b31ff6900d9e","Type":"ContainerDied","Data":"7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb"}
Aug 04 12:04:49.255896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.255883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt" event={"ID":"82e39b82-6dfa-4daf-ad19-b31ff6900d9e","Type":"ContainerDied","Data":"f4ba5a694d2898c9ce1d67801a2d6d0777a0853829686408fd67567fb0c0b4b3"}
Aug 04 12:04:49.263361 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.263321    2558 scope.go:117] "RemoveContainer" containerID="b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8"
Aug 04 12:04:49.263584 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:04:49.263566    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8\": container with ID starting with b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8 not found: ID does not exist" containerID="b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8"
Aug 04 12:04:49.263655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.263602    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8"} err="failed to get container status \"b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8\": rpc error: code = NotFound desc = could not find container \"b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8\": container with ID starting with b97ab40e6758a87c9d2feb78fe3b10093de3aa70653ba4699266db255e46ebc8 not found: ID does not exist"
Aug 04 12:04:49.263655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.263619    2558 scope.go:117] "RemoveContainer" containerID="7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb"
Aug 04 12:04:49.271260 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.271239    2558 scope.go:117] "RemoveContainer" containerID="7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb"
Aug 04 12:04:49.271523 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:04:49.271505    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb\": container with ID starting with 7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb not found: ID does not exist" containerID="7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb"
Aug 04 12:04:49.271583 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.271528    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb"} err="failed to get container status \"7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb\": rpc error: code = NotFound desc = could not find container \"7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb\": container with ID starting with 7c3d0c7ec3e783a70e06d525b9cf8628457bfb722c9ac5d47b32bd475ffaedfb not found: ID does not exist"
Aug 04 12:04:49.298665 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.298639    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"]
Aug 04 12:04:49.310268 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.310245    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b6e21d41-868d8d5675-7bpkt"]
Aug 04 12:04:49.322251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.322200    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdb6q\" (UniqueName: \"kubernetes.io/projected/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-kube-api-access-fdb6q\") pod \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") "
Aug 04 12:04:49.322337 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.322272    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-plugins-config-volume\") pod \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\" (UID: \"b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9\") "
Aug 04 12:04:49.322552 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.322534    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:49.322618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.322557    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:49.322618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.322572    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89z4j\" (UniqueName: \"kubernetes.io/projected/82e39b82-6dfa-4daf-ad19-b31ff6900d9e-kube-api-access-89z4j\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:49.322699 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.322631    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" (UID: "b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:04:49.324264 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.324244    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-kube-api-access-fdb6q" (OuterVolumeSpecName: "kube-api-access-fdb6q") pod "b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" (UID: "b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9"). InnerVolumeSpecName "kube-api-access-fdb6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:04:49.422992 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.422956    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:49.422992 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.422984    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdb6q\" (UniqueName: \"kubernetes.io/projected/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9-kube-api-access-fdb6q\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:04:49.457969 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.457942    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:49.590191 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.590109    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"]
Aug 04 12:04:49.593241 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.593216    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz"]
Aug 04 12:04:49.653704 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.653676    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:49.957002 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:49.956911    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:50.314448 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:50.314413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:51.000544 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:51.000506    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28bf982f-967f-4431-81e0-11346178fb3f" path="/var/lib/kubelet/pods/28bf982f-967f-4431-81e0-11346178fb3f/volumes"
Aug 04 12:04:51.001046 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:51.001025    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82e39b82-6dfa-4daf-ad19-b31ff6900d9e" path="/var/lib/kubelet/pods/82e39b82-6dfa-4daf-ad19-b31ff6900d9e/volumes"
Aug 04 12:04:51.001488 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:51.001473    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" path="/var/lib/kubelet/pods/b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9/volumes"
Aug 04 12:04:52.305128 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:52.305095    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:52.403291 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:52.403237    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:53.372430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372401    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"]
Aug 04 12:04:53.372816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372780    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" containerName="payload-processing"
Aug 04 12:04:53.372816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372793    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" containerName="payload-processing"
Aug 04 12:04:53.372816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372813    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82e39b82-6dfa-4daf-ad19-b31ff6900d9e" containerName="maas-api"
Aug 04 12:04:53.372816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372819    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="82e39b82-6dfa-4daf-ad19-b31ff6900d9e" containerName="maas-api"
Aug 04 12:04:53.372980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372830    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bf982f-967f-4431-81e0-11346178fb3f" containerName="revoke-keys"
Aug 04 12:04:53.372980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372835    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bf982f-967f-4431-81e0-11346178fb3f" containerName="revoke-keys"
Aug 04 12:04:53.372980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372902    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7cbaf32-883a-4fd3-a8d4-b2a0743f2af9" containerName="payload-processing"
Aug 04 12:04:53.372980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372909    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bf982f-967f-4431-81e0-11346178fb3f" containerName="revoke-keys"
Aug 04 12:04:53.372980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.372917    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="82e39b82-6dfa-4daf-ad19-b31ff6900d9e" containerName="maas-api"
Aug 04 12:04:53.377383 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.377364    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.380260 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.380237    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-34f15601-serving-cert\""
Aug 04 12:04:53.388647 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.388620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"]
Aug 04 12:04:53.418723 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.418680    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:53.558416 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.558378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2fa5afc-84d3-4633-a797-48a4956a5d1c-maas-api-tls\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.558608 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.558470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa5afc-84d3-4633-a797-48a4956a5d1c-openshift-service-ca\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.558608 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.558514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vmfb\" (UniqueName: \"kubernetes.io/projected/b2fa5afc-84d3-4633-a797-48a4956a5d1c-kube-api-access-6vmfb\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.625104 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.625029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:53.659224 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.659197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vmfb\" (UniqueName: \"kubernetes.io/projected/b2fa5afc-84d3-4633-a797-48a4956a5d1c-kube-api-access-6vmfb\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.659349 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.659246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2fa5afc-84d3-4633-a797-48a4956a5d1c-maas-api-tls\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.659349 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.659316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa5afc-84d3-4633-a797-48a4956a5d1c-openshift-service-ca\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.659868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.659851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa5afc-84d3-4633-a797-48a4956a5d1c-openshift-service-ca\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.661701 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.661675    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2fa5afc-84d3-4633-a797-48a4956a5d1c-maas-api-tls\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.667561 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.667542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vmfb\" (UniqueName: \"kubernetes.io/projected/b2fa5afc-84d3-4633-a797-48a4956a5d1c-kube-api-access-6vmfb\") pod \"maas-api-e2e-ait-34f15601-76bf6564cc-npjbt\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.688558 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.688539    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:53.822069 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:53.822041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"]
Aug 04 12:04:53.823731 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:04:53.823702    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2fa5afc_84d3_4633_a797_48a4956a5d1c.slice/crio-2f92ccbaeee4d8f212aec312c356c57d378b9fc52b236349524e533d48d58757 WatchSource:0}: Error finding container 2f92ccbaeee4d8f212aec312c356c57d378b9fc52b236349524e533d48d58757: Status 404 returned error can't find the container with id 2f92ccbaeee4d8f212aec312c356c57d378b9fc52b236349524e533d48d58757
Aug 04 12:04:54.011408 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.011378    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:54.117697 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.117662    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"]
Aug 04 12:04:54.119412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.119385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:04:54.122425 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.122407    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.125312 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.125292    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-34f15601\""
Aug 04 12:04:54.125587 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.125568    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-34f15601-dockercfg-vqqkb\""
Aug 04 12:04:54.133342 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.133314    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"]
Aug 04 12:04:54.264858 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.264831    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnvn2\" (UniqueName: \"kubernetes.io/projected/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-kube-api-access-nnvn2\") pod \"payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.264978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.264899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.286186 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.286155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt" event={"ID":"b2fa5afc-84d3-4633-a797-48a4956a5d1c","Type":"ContainerStarted","Data":"2f92ccbaeee4d8f212aec312c356c57d378b9fc52b236349524e533d48d58757"}
Aug 04 12:04:54.365562 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.365513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnvn2\" (UniqueName: \"kubernetes.io/projected/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-kube-api-access-nnvn2\") pod \"payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.365743 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.365602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.366562 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.366537    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.374984 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.374953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnvn2\" (UniqueName: \"kubernetes.io/projected/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-kube-api-access-nnvn2\") pod \"payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.433662 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.433630    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:54.587937 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:54.587911    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"]
Aug 04 12:04:54.589786 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:04:54.589760    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46947dc_b785_4bce_a1e0_6f47bb8ebc6f.slice/crio-7db4752264c155a586a314a24558f37b9a3d656baa816c18b3159872625c2ca0 WatchSource:0}: Error finding container 7db4752264c155a586a314a24558f37b9a3d656baa816c18b3159872625c2ca0: Status 404 returned error can't find the container with id 7db4752264c155a586a314a24558f37b9a3d656baa816c18b3159872625c2ca0
Aug 04 12:04:55.298274 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.298221    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt" event={"ID":"b2fa5afc-84d3-4633-a797-48a4956a5d1c","Type":"ContainerStarted","Data":"ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a"}
Aug 04 12:04:55.298463 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.298437    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:04:55.299781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.299750    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4" event={"ID":"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f","Type":"ContainerStarted","Data":"63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485"}
Aug 04 12:04:55.299918 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.299798    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4" event={"ID":"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f","Type":"ContainerStarted","Data":"7db4752264c155a586a314a24558f37b9a3d656baa816c18b3159872625c2ca0"}
Aug 04 12:04:55.299918 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.299826    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:04:55.319675 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.319628    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt" podStartSLOduration=1.8964357920000001 podStartE2EDuration="2.319614635s" totalImagesPullingTime="423.178843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:04:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:04:55.316739267 +0000 UTC m=+1908.887652074" watchObservedRunningTime="2026-08-04 12:04:55.319614635 +0000 UTC m=+1908.890527468"
Aug 04 12:04:55.335814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:04:55.335775    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4" podStartSLOduration=1.3357619029999999 podStartE2EDuration="1.335761903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:04:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:04:55.334270252 +0000 UTC m=+1908.905183056" watchObservedRunningTime="2026-08-04 12:04:55.335761903 +0000 UTC m=+1908.906674707"
Aug 04 12:05:01.310480 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:01.310446    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:05:06.307256 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:06.307224    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:05:17.986668 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:17.986633    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"]
Aug 04 12:05:17.990347 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:17.990315    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:17.993417 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:17.993398    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:05:17.998950 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:17.998854    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"]
Aug 04 12:05:18.060621 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:18.060591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90513e39-bb4a-437e-ad9e-41532eb5cd92-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-34f15601-dcg7r\" (UID: \"90513e39-bb4a-437e-ad9e-41532eb5cd92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:18.161179 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:18.161148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90513e39-bb4a-437e-ad9e-41532eb5cd92-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-34f15601-dcg7r\" (UID: \"90513e39-bb4a-437e-ad9e-41532eb5cd92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:18.161893 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:18.161873    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90513e39-bb4a-437e-ad9e-41532eb5cd92-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-34f15601-dcg7r\" (UID: \"90513e39-bb4a-437e-ad9e-41532eb5cd92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:18.303180 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:18.303138    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:18.464920 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:18.464893    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"]
Aug 04 12:05:18.466249 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:05:18.466218    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90513e39_bb4a_437e_ad9e_41532eb5cd92.slice/crio-30a30aeb0f1316a405ab6ce1ef7dddfb852febaaa78a7e9f02578e40b9b0ce26 WatchSource:0}: Error finding container 30a30aeb0f1316a405ab6ce1ef7dddfb852febaaa78a7e9f02578e40b9b0ce26: Status 404 returned error can't find the container with id 30a30aeb0f1316a405ab6ce1ef7dddfb852febaaa78a7e9f02578e40b9b0ce26
Aug 04 12:05:19.423683 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:19.423595    2558 generic.go:356] "Generic (PLEG): container finished" podID="90513e39-bb4a-437e-ad9e-41532eb5cd92" containerID="a7a5709d1ea4a04ca2620554908faf412e757a2f6544eafde85b1403fb736a89" exitCode=0
Aug 04 12:05:19.424033 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:19.423687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r" event={"ID":"90513e39-bb4a-437e-ad9e-41532eb5cd92","Type":"ContainerDied","Data":"a7a5709d1ea4a04ca2620554908faf412e757a2f6544eafde85b1403fb736a89"}
Aug 04 12:05:19.424033 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:19.423747    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r" event={"ID":"90513e39-bb4a-437e-ad9e-41532eb5cd92","Type":"ContainerStarted","Data":"30a30aeb0f1316a405ab6ce1ef7dddfb852febaaa78a7e9f02578e40b9b0ce26"}
Aug 04 12:05:20.568487 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:20.568461    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:20.685136 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:20.685028    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90513e39-bb4a-437e-ad9e-41532eb5cd92-maas-api-service-ca\") pod \"90513e39-bb4a-437e-ad9e-41532eb5cd92\" (UID: \"90513e39-bb4a-437e-ad9e-41532eb5cd92\") "
Aug 04 12:05:20.685498 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:20.685476    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90513e39-bb4a-437e-ad9e-41532eb5cd92-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90513e39-bb4a-437e-ad9e-41532eb5cd92" (UID: "90513e39-bb4a-437e-ad9e-41532eb5cd92"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:05:20.786712 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:20.786674    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90513e39-bb4a-437e-ad9e-41532eb5cd92-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:21.435517 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:21.435481    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"
Aug 04 12:05:21.435517 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:21.435501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r" event={"ID":"90513e39-bb4a-437e-ad9e-41532eb5cd92","Type":"ContainerDied","Data":"30a30aeb0f1316a405ab6ce1ef7dddfb852febaaa78a7e9f02578e40b9b0ce26"}
Aug 04 12:05:21.435725 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:21.435548    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30a30aeb0f1316a405ab6ce1ef7dddfb852febaaa78a7e9f02578e40b9b0ce26"
Aug 04 12:05:23.912865 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:23.912819    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"]
Aug 04 12:05:23.913362 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:23.913087    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt" podUID="b2fa5afc-84d3-4633-a797-48a4956a5d1c" containerName="maas-api" containerID="cri-o://ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a" gracePeriod=30
Aug 04 12:05:23.953018 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:23.952985    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:23.974310 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:23.974260    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"]
Aug 04 12:05:23.974537 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:23.974501    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4" podUID="c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" containerName="payload-pre-processing" containerID="cri-o://63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485" gracePeriod=30
Aug 04 12:05:24.052205 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.052173    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:24.120201 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.120166    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"]
Aug 04 12:05:24.122120 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.122098    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-34f15601-dcg7r"]
Aug 04 12:05:24.130299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.130253    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:24.259336 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.259309    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:05:24.262441 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.262414    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:05:24.316412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.316379    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-plugins-config-volume\") pod \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") "
Aug 04 12:05:24.316589 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.316509    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2fa5afc-84d3-4633-a797-48a4956a5d1c-maas-api-tls\") pod \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") "
Aug 04 12:05:24.316589 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.316559    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vmfb\" (UniqueName: \"kubernetes.io/projected/b2fa5afc-84d3-4633-a797-48a4956a5d1c-kube-api-access-6vmfb\") pod \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") "
Aug 04 12:05:24.316711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.316591    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnvn2\" (UniqueName: \"kubernetes.io/projected/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-kube-api-access-nnvn2\") pod \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\" (UID: \"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f\") "
Aug 04 12:05:24.316711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.316636    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa5afc-84d3-4633-a797-48a4956a5d1c-openshift-service-ca\") pod \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\" (UID: \"b2fa5afc-84d3-4633-a797-48a4956a5d1c\") "
Aug 04 12:05:24.316873 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.316828    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" (UID: "c46947dc-b785-4bce-a1e0-6f47bb8ebc6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:05:24.317168 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.317148    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2fa5afc-84d3-4633-a797-48a4956a5d1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2fa5afc-84d3-4633-a797-48a4956a5d1c" (UID: "b2fa5afc-84d3-4633-a797-48a4956a5d1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:05:24.318732 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.318709    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2fa5afc-84d3-4633-a797-48a4956a5d1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2fa5afc-84d3-4633-a797-48a4956a5d1c" (UID: "b2fa5afc-84d3-4633-a797-48a4956a5d1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:05:24.319043 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.319027    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2fa5afc-84d3-4633-a797-48a4956a5d1c-kube-api-access-6vmfb" (OuterVolumeSpecName: "kube-api-access-6vmfb") pod "b2fa5afc-84d3-4633-a797-48a4956a5d1c" (UID: "b2fa5afc-84d3-4633-a797-48a4956a5d1c"). InnerVolumeSpecName "kube-api-access-6vmfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:05:24.319163 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.319141    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-kube-api-access-nnvn2" (OuterVolumeSpecName: "kube-api-access-nnvn2") pod "c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" (UID: "c46947dc-b785-4bce-a1e0-6f47bb8ebc6f"). InnerVolumeSpecName "kube-api-access-nnvn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:05:24.417983 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.417952    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2fa5afc-84d3-4633-a797-48a4956a5d1c-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:24.417983 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.417981    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vmfb\" (UniqueName: \"kubernetes.io/projected/b2fa5afc-84d3-4633-a797-48a4956a5d1c-kube-api-access-6vmfb\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:24.417983 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.417993    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnvn2\" (UniqueName: \"kubernetes.io/projected/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-kube-api-access-nnvn2\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:24.418220 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.418002    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa5afc-84d3-4633-a797-48a4956a5d1c-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:24.418220 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.418010    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:24.451174 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.451092    2558 generic.go:356] "Generic (PLEG): container finished" podID="b2fa5afc-84d3-4633-a797-48a4956a5d1c" containerID="ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a" exitCode=0
Aug 04 12:05:24.451174 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.451144    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"
Aug 04 12:05:24.451174 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.451138    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt" event={"ID":"b2fa5afc-84d3-4633-a797-48a4956a5d1c","Type":"ContainerDied","Data":"ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a"}
Aug 04 12:05:24.451465 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.451196    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt" event={"ID":"b2fa5afc-84d3-4633-a797-48a4956a5d1c","Type":"ContainerDied","Data":"2f92ccbaeee4d8f212aec312c356c57d378b9fc52b236349524e533d48d58757"}
Aug 04 12:05:24.451465 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.451218    2558 scope.go:117] "RemoveContainer" containerID="ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a"
Aug 04 12:05:24.452629 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.452604    2558 generic.go:356] "Generic (PLEG): container finished" podID="c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" containerID="63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485" exitCode=0
Aug 04 12:05:24.452729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.452636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4" event={"ID":"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f","Type":"ContainerDied","Data":"63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485"}
Aug 04 12:05:24.452729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.452653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"
Aug 04 12:05:24.452729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.452690    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4" event={"ID":"c46947dc-b785-4bce-a1e0-6f47bb8ebc6f","Type":"ContainerDied","Data":"7db4752264c155a586a314a24558f37b9a3d656baa816c18b3159872625c2ca0"}
Aug 04 12:05:24.461684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.461667    2558 scope.go:117] "RemoveContainer" containerID="ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a"
Aug 04 12:05:24.462054 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:05:24.462034    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a\": container with ID starting with ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a not found: ID does not exist" containerID="ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a"
Aug 04 12:05:24.462135 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.462061    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a"} err="failed to get container status \"ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a\": rpc error: code = NotFound desc = could not find container \"ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a\": container with ID starting with ed031deff88c247f234420fe6a4dd5d01d7ae8e50619854948ddeaac79c9235a not found: ID does not exist"
Aug 04 12:05:24.462135 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.462089    2558 scope.go:117] "RemoveContainer" containerID="63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485"
Aug 04 12:05:24.470378 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.470360    2558 scope.go:117] "RemoveContainer" containerID="63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485"
Aug 04 12:05:24.470599 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:05:24.470580    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485\": container with ID starting with 63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485 not found: ID does not exist" containerID="63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485"
Aug 04 12:05:24.470654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.470605    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485"} err="failed to get container status \"63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485\": rpc error: code = NotFound desc = could not find container \"63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485\": container with ID starting with 63a7b2d72ba6b06d171a90dc4c7dc7d088ab6633fceb3e0a0267034526507485 not found: ID does not exist"
Aug 04 12:05:24.474290 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.474258    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"]
Aug 04 12:05:24.478039 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.478018    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-34f15601-76bf6564cc-npjbt"]
Aug 04 12:05:24.491093 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.491069    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"]
Aug 04 12:05:24.495477 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.495456    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4"]
Aug 04 12:05:24.994114 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.994073    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90513e39-bb4a-437e-ad9e-41532eb5cd92" path="/var/lib/kubelet/pods/90513e39-bb4a-437e-ad9e-41532eb5cd92/volumes"
Aug 04 12:05:24.994672 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.994652    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2fa5afc-84d3-4633-a797-48a4956a5d1c" path="/var/lib/kubelet/pods/b2fa5afc-84d3-4633-a797-48a4956a5d1c/volumes"
Aug 04 12:05:24.995038 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:24.995020    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" path="/var/lib/kubelet/pods/c46947dc-b785-4bce-a1e0-6f47bb8ebc6f/volumes"
Aug 04 12:05:25.263946 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:25.263915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:25.496471 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:25.496437    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:25.697488 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:25.697405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:25.988649 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:25.988573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:26.228344 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:26.228312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:28.476868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:28.476837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:28.593402 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:28.593371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:29.587131 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587100    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"]
Aug 04 12:05:29.587531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587466    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2fa5afc-84d3-4633-a797-48a4956a5d1c" containerName="maas-api"
Aug 04 12:05:29.587531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587481    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2fa5afc-84d3-4633-a797-48a4956a5d1c" containerName="maas-api"
Aug 04 12:05:29.587531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587512    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90513e39-bb4a-437e-ad9e-41532eb5cd92" containerName="revoke-keys"
Aug 04 12:05:29.587531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587518    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="90513e39-bb4a-437e-ad9e-41532eb5cd92" containerName="revoke-keys"
Aug 04 12:05:29.587531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587526    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" containerName="payload-pre-processing"
Aug 04 12:05:29.587531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587532    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" containerName="payload-pre-processing"
Aug 04 12:05:29.587750 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587591    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c46947dc-b785-4bce-a1e0-6f47bb8ebc6f" containerName="payload-pre-processing"
Aug 04 12:05:29.587750 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587599    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="90513e39-bb4a-437e-ad9e-41532eb5cd92" containerName="revoke-keys"
Aug 04 12:05:29.587750 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.587606    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2fa5afc-84d3-4633-a797-48a4956a5d1c" containerName="maas-api"
Aug 04 12:05:29.592201 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.592180    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.595421 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.595402    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b9abcfc3-serving-cert\""
Aug 04 12:05:29.610710 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.610691    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"]
Aug 04 12:05:29.635491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.635461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:29.664209 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.664180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7282b777-017a-4ebd-a865-d670d849611d-openshift-service-ca\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.664349 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.664240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chzg5\" (UniqueName: \"kubernetes.io/projected/7282b777-017a-4ebd-a865-d670d849611d-kube-api-access-chzg5\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.664349 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.664315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.765720 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.765684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chzg5\" (UniqueName: \"kubernetes.io/projected/7282b777-017a-4ebd-a865-d670d849611d-kube-api-access-chzg5\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.765881 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.765734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.765881 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.765773    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7282b777-017a-4ebd-a865-d670d849611d-openshift-service-ca\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.765953 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:05:29.765903    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-serving-cert: secret "maas-api-e2e-derive-b9abcfc3-serving-cert" not found
Aug 04 12:05:29.765985 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:05:29.765974    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls podName:7282b777-017a-4ebd-a865-d670d849611d nodeName:}" failed. No retries permitted until 2026-08-04 12:05:30.265950988 +0000 UTC m=+1943.836863782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls") pod "maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" (UID: "7282b777-017a-4ebd-a865-d670d849611d") : secret "maas-api-e2e-derive-b9abcfc3-serving-cert" not found
Aug 04 12:05:29.766465 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.766448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7282b777-017a-4ebd-a865-d670d849611d-openshift-service-ca\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:29.778507 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:29.778483    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chzg5\" (UniqueName: \"kubernetes.io/projected/7282b777-017a-4ebd-a865-d670d849611d-kube-api-access-chzg5\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:30.074396 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.074358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:30.270143 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.270108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:30.272651 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.272627    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls\") pod \"maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:30.410812 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.410735    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"]
Aug 04 12:05:30.414481 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.414459    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.417779 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.417757    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b9abcfc3-dockercfg-7p8gv\""
Aug 04 12:05:30.418071 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.418053    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b9abcfc3\""
Aug 04 12:05:30.425008 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.424982    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"]
Aug 04 12:05:30.471841 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.471807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c817c951-7982-4d76-941f-416cb0d74b2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.471985 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.471931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t7r6\" (UniqueName: \"kubernetes.io/projected/c817c951-7982-4d76-941f-416cb0d74b2c-kube-api-access-6t7r6\") pod \"payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.502203 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.502174    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:30.573007 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.572963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c817c951-7982-4d76-941f-416cb0d74b2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.573179 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.573056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t7r6\" (UniqueName: \"kubernetes.io/projected/c817c951-7982-4d76-941f-416cb0d74b2c-kube-api-access-6t7r6\") pod \"payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.574427 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.574396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c817c951-7982-4d76-941f-416cb0d74b2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.582227 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.582198    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t7r6\" (UniqueName: \"kubernetes.io/projected/c817c951-7982-4d76-941f-416cb0d74b2c-kube-api-access-6t7r6\") pod \"payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.727062 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.726982    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:30.857809 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.857783    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"]
Aug 04 12:05:30.859018 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:05:30.858994    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7282b777_017a_4ebd_a865_d670d849611d.slice/crio-08742ba61e48342b81aa84d2637c5228e81cb55882ba50c97eeaa7d9428e5a06 WatchSource:0}: Error finding container 08742ba61e48342b81aa84d2637c5228e81cb55882ba50c97eeaa7d9428e5a06: Status 404 returned error can't find the container with id 08742ba61e48342b81aa84d2637c5228e81cb55882ba50c97eeaa7d9428e5a06
Aug 04 12:05:30.865991 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:30.865964    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"]
Aug 04 12:05:30.866982 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:05:30.866957    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc817c951_7982_4d76_941f_416cb0d74b2c.slice/crio-0850b4ddcd854f927ea31d1bb5131f2d0529173e7318108598aa21155fe502e3 WatchSource:0}: Error finding container 0850b4ddcd854f927ea31d1bb5131f2d0529173e7318108598aa21155fe502e3: Status 404 returned error can't find the container with id 0850b4ddcd854f927ea31d1bb5131f2d0529173e7318108598aa21155fe502e3
Aug 04 12:05:31.493529 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.493488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" event={"ID":"7282b777-017a-4ebd-a865-d670d849611d","Type":"ContainerStarted","Data":"08742ba61e48342b81aa84d2637c5228e81cb55882ba50c97eeaa7d9428e5a06"}
Aug 04 12:05:31.495026 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.494994    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m" event={"ID":"c817c951-7982-4d76-941f-416cb0d74b2c","Type":"ContainerStarted","Data":"b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399"}
Aug 04 12:05:31.495163 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.495045    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m" event={"ID":"c817c951-7982-4d76-941f-416cb0d74b2c","Type":"ContainerStarted","Data":"0850b4ddcd854f927ea31d1bb5131f2d0529173e7318108598aa21155fe502e3"}
Aug 04 12:05:31.495163 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.495137    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:31.518360 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.518307    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m" podStartSLOduration=1.518275235 podStartE2EDuration="1.518275235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:05:31.515437629 +0000 UTC m=+1945.086350435" watchObservedRunningTime="2026-08-04 12:05:31.518275235 +0000 UTC m=+1945.089188039"
Aug 04 12:05:31.632046 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.632012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:31.703202 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:31.703168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:05:32.501857 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:32.501823    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" event={"ID":"7282b777-017a-4ebd-a865-d670d849611d","Type":"ContainerStarted","Data":"a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251"}
Aug 04 12:05:32.502346 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:32.501980    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:32.524824 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:32.524777    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" podStartSLOduration=2.61030291 podStartE2EDuration="3.52475706s" totalImagesPullingTime="914.45415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:05:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:05:32.523140365 +0000 UTC m=+1946.094053169" watchObservedRunningTime="2026-08-04 12:05:32.52475706 +0000 UTC m=+1946.095669863"
Aug 04 12:05:38.512154 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:38.512123    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:05:42.504634 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:42.504600    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:05:53.393177 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.393136    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"]
Aug 04 12:05:53.396936 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.396913    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:53.405415 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.405383    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:05:53.409047 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.409019    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"]
Aug 04 12:05:53.583447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.583404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f645054-6862-4311-8e91-31dd67b61f34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms\" (UID: \"1f645054-6862-4311-8e91-31dd67b61f34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:53.684354 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.684243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f645054-6862-4311-8e91-31dd67b61f34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms\" (UID: \"1f645054-6862-4311-8e91-31dd67b61f34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:53.684856 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.684838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f645054-6862-4311-8e91-31dd67b61f34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms\" (UID: \"1f645054-6862-4311-8e91-31dd67b61f34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:53.717338 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.717309    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:53.853214 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:53.853184    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"]
Aug 04 12:05:53.854740 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:05:53.854693    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f645054_6862_4311_8e91_31dd67b61f34.slice/crio-a90e2c3a2d5496155159c4585d9576d9aa831821831c10b7dabfeb6063c43639 WatchSource:0}: Error finding container a90e2c3a2d5496155159c4585d9576d9aa831821831c10b7dabfeb6063c43639: Status 404 returned error can't find the container with id a90e2c3a2d5496155159c4585d9576d9aa831821831c10b7dabfeb6063c43639
Aug 04 12:05:54.615637 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:54.615602    2558 generic.go:356] "Generic (PLEG): container finished" podID="1f645054-6862-4311-8e91-31dd67b61f34" containerID="8062baddf3382d71123f5e823302fc7eb55b19c9a3250ea12c6452caf4101fbc" exitCode=0
Aug 04 12:05:54.616003 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:54.615641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms" event={"ID":"1f645054-6862-4311-8e91-31dd67b61f34","Type":"ContainerDied","Data":"8062baddf3382d71123f5e823302fc7eb55b19c9a3250ea12c6452caf4101fbc"}
Aug 04 12:05:54.616003 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:54.615710    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms" event={"ID":"1f645054-6862-4311-8e91-31dd67b61f34","Type":"ContainerStarted","Data":"a90e2c3a2d5496155159c4585d9576d9aa831821831c10b7dabfeb6063c43639"}
Aug 04 12:05:55.754878 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:55.754856    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:55.799590 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:55.799563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f645054-6862-4311-8e91-31dd67b61f34-maas-api-service-ca\") pod \"1f645054-6862-4311-8e91-31dd67b61f34\" (UID: \"1f645054-6862-4311-8e91-31dd67b61f34\") "
Aug 04 12:05:55.799935 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:55.799912    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f645054-6862-4311-8e91-31dd67b61f34-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f645054-6862-4311-8e91-31dd67b61f34" (UID: "1f645054-6862-4311-8e91-31dd67b61f34"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:05:55.900356 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:55.900263    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f645054-6862-4311-8e91-31dd67b61f34-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:05:56.627655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:56.627624    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"
Aug 04 12:05:56.627655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:56.627644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms" event={"ID":"1f645054-6862-4311-8e91-31dd67b61f34","Type":"ContainerDied","Data":"a90e2c3a2d5496155159c4585d9576d9aa831821831c10b7dabfeb6063c43639"}
Aug 04 12:05:56.627882 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:56.627687    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a90e2c3a2d5496155159c4585d9576d9aa831821831c10b7dabfeb6063c43639"
Aug 04 12:05:59.612675 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:05:59.612593    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:00.224814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.224780    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"]
Aug 04 12:06:00.225064 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.225037    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" podUID="7282b777-017a-4ebd-a865-d670d849611d" containerName="maas-api" containerID="cri-o://a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251" gracePeriod=30
Aug 04 12:06:00.289987 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.289955    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"]
Aug 04 12:06:00.290517 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.290486    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m" podUID="c817c951-7982-4d76-941f-416cb0d74b2c" containerName="payload-pre-processing" containerID="cri-o://b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399" gracePeriod=30
Aug 04 12:06:00.419059 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.419031    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"]
Aug 04 12:06:00.423940 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.423912    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b9abcfc3-jtdms"]
Aug 04 12:06:00.535069 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.535046    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:06:00.570315 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.570274    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:06:00.634508 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634475    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls\") pod \"7282b777-017a-4ebd-a865-d670d849611d\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") "
Aug 04 12:06:00.634956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634532    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7282b777-017a-4ebd-a865-d670d849611d-openshift-service-ca\") pod \"7282b777-017a-4ebd-a865-d670d849611d\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") "
Aug 04 12:06:00.634956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634552    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c817c951-7982-4d76-941f-416cb0d74b2c-plugins-config-volume\") pod \"c817c951-7982-4d76-941f-416cb0d74b2c\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") "
Aug 04 12:06:00.634956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634656    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t7r6\" (UniqueName: \"kubernetes.io/projected/c817c951-7982-4d76-941f-416cb0d74b2c-kube-api-access-6t7r6\") pod \"c817c951-7982-4d76-941f-416cb0d74b2c\" (UID: \"c817c951-7982-4d76-941f-416cb0d74b2c\") "
Aug 04 12:06:00.634956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634718    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chzg5\" (UniqueName: \"kubernetes.io/projected/7282b777-017a-4ebd-a865-d670d849611d-kube-api-access-chzg5\") pod \"7282b777-017a-4ebd-a865-d670d849611d\" (UID: \"7282b777-017a-4ebd-a865-d670d849611d\") "
Aug 04 12:06:00.634956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634895    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7282b777-017a-4ebd-a865-d670d849611d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7282b777-017a-4ebd-a865-d670d849611d" (UID: "7282b777-017a-4ebd-a865-d670d849611d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:06:00.634956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.634927    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c817c951-7982-4d76-941f-416cb0d74b2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c817c951-7982-4d76-941f-416cb0d74b2c" (UID: "c817c951-7982-4d76-941f-416cb0d74b2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:06:00.635246 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.635089    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7282b777-017a-4ebd-a865-d670d849611d-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:06:00.635246 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.635110    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c817c951-7982-4d76-941f-416cb0d74b2c-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:06:00.636695 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.636670    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c817c951-7982-4d76-941f-416cb0d74b2c-kube-api-access-6t7r6" (OuterVolumeSpecName: "kube-api-access-6t7r6") pod "c817c951-7982-4d76-941f-416cb0d74b2c" (UID: "c817c951-7982-4d76-941f-416cb0d74b2c"). InnerVolumeSpecName "kube-api-access-6t7r6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:06:00.636748 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.636705    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7282b777-017a-4ebd-a865-d670d849611d" (UID: "7282b777-017a-4ebd-a865-d670d849611d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:06:00.636785 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.636771    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7282b777-017a-4ebd-a865-d670d849611d-kube-api-access-chzg5" (OuterVolumeSpecName: "kube-api-access-chzg5") pod "7282b777-017a-4ebd-a865-d670d849611d" (UID: "7282b777-017a-4ebd-a865-d670d849611d"). InnerVolumeSpecName "kube-api-access-chzg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:06:00.650247 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.650220    2558 generic.go:356] "Generic (PLEG): container finished" podID="c817c951-7982-4d76-941f-416cb0d74b2c" containerID="b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399" exitCode=0
Aug 04 12:06:00.650383 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.650304    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"
Aug 04 12:06:00.650383 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.650314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m" event={"ID":"c817c951-7982-4d76-941f-416cb0d74b2c","Type":"ContainerDied","Data":"b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399"}
Aug 04 12:06:00.650503 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.650391    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m" event={"ID":"c817c951-7982-4d76-941f-416cb0d74b2c","Type":"ContainerDied","Data":"0850b4ddcd854f927ea31d1bb5131f2d0529173e7318108598aa21155fe502e3"}
Aug 04 12:06:00.650503 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.650424    2558 scope.go:117] "RemoveContainer" containerID="b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399"
Aug 04 12:06:00.651625 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.651607    2558 generic.go:356] "Generic (PLEG): container finished" podID="7282b777-017a-4ebd-a865-d670d849611d" containerID="a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251" exitCode=0
Aug 04 12:06:00.651714 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.651646    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" event={"ID":"7282b777-017a-4ebd-a865-d670d849611d","Type":"ContainerDied","Data":"a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251"}
Aug 04 12:06:00.651714 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.651687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv" event={"ID":"7282b777-017a-4ebd-a865-d670d849611d","Type":"ContainerDied","Data":"08742ba61e48342b81aa84d2637c5228e81cb55882ba50c97eeaa7d9428e5a06"}
Aug 04 12:06:00.651714 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.651653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"
Aug 04 12:06:00.663173 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.663145    2558 scope.go:117] "RemoveContainer" containerID="b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399"
Aug 04 12:06:00.663476 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:06:00.663456    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399\": container with ID starting with b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399 not found: ID does not exist" containerID="b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399"
Aug 04 12:06:00.663572 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.663482    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399"} err="failed to get container status \"b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399\": rpc error: code = NotFound desc = could not find container \"b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399\": container with ID starting with b534694a7051aaa738d3f0ed4673ae0d569fb82247bc3114a28b7b48cecb8399 not found: ID does not exist"
Aug 04 12:06:00.663572 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.663499    2558 scope.go:117] "RemoveContainer" containerID="a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251"
Aug 04 12:06:00.671711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.671691    2558 scope.go:117] "RemoveContainer" containerID="a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251"
Aug 04 12:06:00.671940 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:06:00.671920    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251\": container with ID starting with a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251 not found: ID does not exist" containerID="a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251"
Aug 04 12:06:00.671988 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.671947    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251"} err="failed to get container status \"a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251\": rpc error: code = NotFound desc = could not find container \"a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251\": container with ID starting with a09947ac417249938c650b178e20c441aca49267476edae7b0f888a8505bb251 not found: ID does not exist"
Aug 04 12:06:00.677973 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.677950    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"]
Aug 04 12:06:00.680548 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.680529    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m"]
Aug 04 12:06:00.693502 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.693481    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"]
Aug 04 12:06:00.698343 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.698324    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b9abcfc3-79d66495d9-h74dv"]
Aug 04 12:06:00.736470 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.736447    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chzg5\" (UniqueName: \"kubernetes.io/projected/7282b777-017a-4ebd-a865-d670d849611d-kube-api-access-chzg5\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:06:00.736470 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.736468    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7282b777-017a-4ebd-a865-d670d849611d-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:06:00.736599 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.736480    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t7r6\" (UniqueName: \"kubernetes.io/projected/c817c951-7982-4d76-941f-416cb0d74b2c-kube-api-access-6t7r6\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:06:00.993863 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.993784    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f645054-6862-4311-8e91-31dd67b61f34" path="/var/lib/kubelet/pods/1f645054-6862-4311-8e91-31dd67b61f34/volumes"
Aug 04 12:06:00.994169 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.994156    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7282b777-017a-4ebd-a865-d670d849611d" path="/var/lib/kubelet/pods/7282b777-017a-4ebd-a865-d670d849611d/volumes"
Aug 04 12:06:00.994505 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:00.994493    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c817c951-7982-4d76-941f-416cb0d74b2c" path="/var/lib/kubelet/pods/c817c951-7982-4d76-941f-416cb0d74b2c/volumes"
Aug 04 12:06:03.086429 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:03.086396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:14.854636 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:14.854600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:20.875872 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:20.875834    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:20.990339 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:20.990308    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:21.080488 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:21.080452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:21.190805 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:21.190720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:22.281029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:22.280994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:23.799023 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:23.798995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:47.500054 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:47.500020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:53.621349 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:53.621312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:53.727002 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:53.726962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:06:53.821542 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:06:53.821499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:09.684373 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:09.684339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:15.376973 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:15.376939    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:15.478886 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:15.478856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:15.577565 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:15.577529    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:26.143538 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:26.143506    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:37.231928 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:37.231843    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:43.086733 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:43.086702    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:43.199071 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:43.199033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:43.287726 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:43.287691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:43.388041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:43.387966    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:44.039041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:44.039004    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:44.139169 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:44.139138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:44.238820 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:44.238787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:45.848578 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:45.848532    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:51.291379 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:51.291342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:52.400051 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:52.400018    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:52.500816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:52.500784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:53.141336 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:53.141272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:59.640297 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:59.640229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:59.742453 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:59.742405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:07:59.976514 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:07:59.976425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:00.242101 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:00.242016    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:00.560145 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:00.560117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:03.351558 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:03.351520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:03.447423 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:03.447388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:04.435271 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435237    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"]
Aug 04 12:08:04.435654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435601    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f645054-6862-4311-8e91-31dd67b61f34" containerName="revoke-keys"
Aug 04 12:08:04.435654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435614    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f645054-6862-4311-8e91-31dd67b61f34" containerName="revoke-keys"
Aug 04 12:08:04.435654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435631    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7282b777-017a-4ebd-a865-d670d849611d" containerName="maas-api"
Aug 04 12:08:04.435654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435637    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7282b777-017a-4ebd-a865-d670d849611d" containerName="maas-api"
Aug 04 12:08:04.435654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435646    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c817c951-7982-4d76-941f-416cb0d74b2c" containerName="payload-pre-processing"
Aug 04 12:08:04.435654 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435652    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c817c951-7982-4d76-941f-416cb0d74b2c" containerName="payload-pre-processing"
Aug 04 12:08:04.435848 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435717    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f645054-6862-4311-8e91-31dd67b61f34" containerName="revoke-keys"
Aug 04 12:08:04.435848 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435725    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7282b777-017a-4ebd-a865-d670d849611d" containerName="maas-api"
Aug 04 12:08:04.435848 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.435731    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c817c951-7982-4d76-941f-416cb0d74b2c" containerName="payload-pre-processing"
Aug 04 12:08:04.439074 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.439054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.441655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.441636    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6958d802-serving-cert\""
Aug 04 12:08:04.451554 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.451520    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"]
Aug 04 12:08:04.473318 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.473292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:04.529753 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.529729    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-maas-api-tls\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.529906 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.529784    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bspz\" (UniqueName: \"kubernetes.io/projected/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-kube-api-access-2bspz\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.529906 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.529884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-openshift-service-ca\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.575695 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.575667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:04.630761 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.630728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bspz\" (UniqueName: \"kubernetes.io/projected/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-kube-api-access-2bspz\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.630940 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.630782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-openshift-service-ca\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.630940 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.630818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-maas-api-tls\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.631568 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.631548    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-openshift-service-ca\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.633266 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.633244    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-maas-api-tls\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.639665 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.639647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bspz\" (UniqueName: \"kubernetes.io/projected/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-kube-api-access-2bspz\") pod \"maas-api-e2e-mt-6958d802-d79974ddd-nrh4r\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.753040 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.753017    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:04.975486 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:04.975446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:05.075957 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.075846    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:05.108799 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.108767    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"]
Aug 04 12:08:05.112946 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.112924    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"]
Aug 04 12:08:05.113064 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.113045    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.114500 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:08:05.114479    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fc735c_c631_4eef_9e7a_a9d14d8b7edf.slice/crio-8680d8058b34a4efae4b1dbdc318330b741b09ff7f1c49da191f741a8b9bea98 WatchSource:0}: Error finding container 8680d8058b34a4efae4b1dbdc318330b741b09ff7f1c49da191f741a8b9bea98: Status 404 returned error can't find the container with id 8680d8058b34a4efae4b1dbdc318330b741b09ff7f1c49da191f741a8b9bea98
Aug 04 12:08:05.115863 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.115843    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6958d802\""
Aug 04 12:08:05.115945 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.115907    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6958d802-dockercfg-g8l8j\""
Aug 04 12:08:05.123008 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.122981    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"]
Aug 04 12:08:05.237592 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.237560    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a53b3b83-e00e-4630-9975-ce4728713836-plugins-config-volume\") pod \"payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") " pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.237820 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.237661    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69tvw\" (UniqueName: \"kubernetes.io/projected/a53b3b83-e00e-4630-9975-ce4728713836-kube-api-access-69tvw\") pod \"payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") " pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.308923 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.308889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r" event={"ID":"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf","Type":"ContainerStarted","Data":"8680d8058b34a4efae4b1dbdc318330b741b09ff7f1c49da191f741a8b9bea98"}
Aug 04 12:08:05.338534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.338455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a53b3b83-e00e-4630-9975-ce4728713836-plugins-config-volume\") pod \"payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") " pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.338534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.338490    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69tvw\" (UniqueName: \"kubernetes.io/projected/a53b3b83-e00e-4630-9975-ce4728713836-kube-api-access-69tvw\") pod \"payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") " pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.339186 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.339165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a53b3b83-e00e-4630-9975-ce4728713836-plugins-config-volume\") pod \"payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") " pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.365842 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.365815    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69tvw\" (UniqueName: \"kubernetes.io/projected/a53b3b83-e00e-4630-9975-ce4728713836-kube-api-access-69tvw\") pod \"payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") " pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.433439 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.433409    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:05.562563 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:05.562540    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"]
Aug 04 12:08:05.564477 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:08:05.564447    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda53b3b83_e00e_4630_9975_ce4728713836.slice/crio-9c751aa660e1c16dda233465e7b14d63cf0de7be36dd26531fa5bf1b88937d78 WatchSource:0}: Error finding container 9c751aa660e1c16dda233465e7b14d63cf0de7be36dd26531fa5bf1b88937d78: Status 404 returned error can't find the container with id 9c751aa660e1c16dda233465e7b14d63cf0de7be36dd26531fa5bf1b88937d78
Aug 04 12:08:06.315913 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.315879    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r" event={"ID":"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf","Type":"ContainerStarted","Data":"6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767"}
Aug 04 12:08:06.316128 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.316053    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:06.317461 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.317438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js" event={"ID":"a53b3b83-e00e-4630-9975-ce4728713836","Type":"ContainerStarted","Data":"92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595"}
Aug 04 12:08:06.317591 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.317480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js" event={"ID":"a53b3b83-e00e-4630-9975-ce4728713836","Type":"ContainerStarted","Data":"9c751aa660e1c16dda233465e7b14d63cf0de7be36dd26531fa5bf1b88937d78"}
Aug 04 12:08:06.317591 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.317508    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:06.333353 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.333310    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r" podStartSLOduration=1.849605691 podStartE2EDuration="2.33327291s" totalImagesPullingTime="483.667219ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:08:06.332362006 +0000 UTC m=+2099.903274811" watchObservedRunningTime="2026-08-04 12:08:06.33327291 +0000 UTC m=+2099.904185714"
Aug 04 12:08:06.351532 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:06.351470    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js" podStartSLOduration=1.351450561 podStartE2EDuration="1.351450561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:08:06.34762139 +0000 UTC m=+2099.918534197" watchObservedRunningTime="2026-08-04 12:08:06.351450561 +0000 UTC m=+2099.922363369"
Aug 04 12:08:07.055553 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:07.055524    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:08:07.062208 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:07.062184    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:08:12.328693 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:12.328664    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:17.326014 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:17.325986    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:28.319514 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:28.319478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:28.983165 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:28.983136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:29.083723 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:29.083680    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:29.502982 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:29.502944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:35.043465 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.043429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:35.143623 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.143587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:35.412299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.412210    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"]
Aug 04 12:08:35.415632 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.415616    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:35.418854 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.418833    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:08:35.426208 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.426187    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"]
Aug 04 12:08:35.504768 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.504737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24323c-71d9-4064-bda0-f0cda9c8eb32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6958d802-bvzl8\" (UID: \"4a24323c-71d9-4064-bda0-f0cda9c8eb32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:35.605274 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.605243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24323c-71d9-4064-bda0-f0cda9c8eb32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6958d802-bvzl8\" (UID: \"4a24323c-71d9-4064-bda0-f0cda9c8eb32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:35.605875 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.605856    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24323c-71d9-4064-bda0-f0cda9c8eb32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6958d802-bvzl8\" (UID: \"4a24323c-71d9-4064-bda0-f0cda9c8eb32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:35.730449 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.730371    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:35.866978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:35.866946    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"]
Aug 04 12:08:35.868334 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:08:35.868301    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a24323c_71d9_4064_bda0_f0cda9c8eb32.slice/crio-b3153a73d76785c3bdc5ef45345603f830f9190682030627339e93b6afc2201d WatchSource:0}: Error finding container b3153a73d76785c3bdc5ef45345603f830f9190682030627339e93b6afc2201d: Status 404 returned error can't find the container with id b3153a73d76785c3bdc5ef45345603f830f9190682030627339e93b6afc2201d
Aug 04 12:08:36.481594 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:36.481502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8" event={"ID":"4a24323c-71d9-4064-bda0-f0cda9c8eb32","Type":"ContainerStarted","Data":"141da40da5a80ffe4fc9c8081108582a379ab85e21cc4b67f2ee95cae0623564"}
Aug 04 12:08:36.481594 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:36.481583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8" event={"ID":"4a24323c-71d9-4064-bda0-f0cda9c8eb32","Type":"ContainerStarted","Data":"b3153a73d76785c3bdc5ef45345603f830f9190682030627339e93b6afc2201d"}
Aug 04 12:08:36.497245 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:36.497199    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8" podStartSLOduration=1.49718632 podStartE2EDuration="1.49718632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:08:36.495870481 +0000 UTC m=+2130.066783296" watchObservedRunningTime="2026-08-04 12:08:36.49718632 +0000 UTC m=+2130.068099123"
Aug 04 12:08:37.488706 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:37.488660    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a24323c-71d9-4064-bda0-f0cda9c8eb32" containerID="141da40da5a80ffe4fc9c8081108582a379ab85e21cc4b67f2ee95cae0623564" exitCode=0
Aug 04 12:08:37.489177 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:37.488712    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8" event={"ID":"4a24323c-71d9-4064-bda0-f0cda9c8eb32","Type":"ContainerDied","Data":"141da40da5a80ffe4fc9c8081108582a379ab85e21cc4b67f2ee95cae0623564"}
Aug 04 12:08:38.621848 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:38.621820    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:38.736190 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:38.736158    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24323c-71d9-4064-bda0-f0cda9c8eb32-maas-api-service-ca\") pod \"4a24323c-71d9-4064-bda0-f0cda9c8eb32\" (UID: \"4a24323c-71d9-4064-bda0-f0cda9c8eb32\") "
Aug 04 12:08:38.736583 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:38.736557    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a24323c-71d9-4064-bda0-f0cda9c8eb32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a24323c-71d9-4064-bda0-f0cda9c8eb32" (UID: "4a24323c-71d9-4064-bda0-f0cda9c8eb32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:08:38.837319 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:38.837272    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24323c-71d9-4064-bda0-f0cda9c8eb32-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:08:39.500365 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:39.500332    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"
Aug 04 12:08:39.500365 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:39.500358    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8" event={"ID":"4a24323c-71d9-4064-bda0-f0cda9c8eb32","Type":"ContainerDied","Data":"b3153a73d76785c3bdc5ef45345603f830f9190682030627339e93b6afc2201d"}
Aug 04 12:08:39.500601 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:39.500417    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3153a73d76785c3bdc5ef45345603f830f9190682030627339e93b6afc2201d"
Aug 04 12:08:45.797622 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.797592    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"]
Aug 04 12:08:45.798018 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.797796    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r" podUID="b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" containerName="maas-api" containerID="cri-o://6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767" gracePeriod=30
Aug 04 12:08:45.829015 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.828987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:45.843931 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.843900    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"]
Aug 04 12:08:45.844144 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.844122    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js" podUID="a53b3b83-e00e-4630-9975-ce4728713836" containerName="payload-processing" containerID="cri-o://92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595" gracePeriod=30
Aug 04 12:08:45.926966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.926920    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:45.985228 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.985194    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"]
Aug 04 12:08:45.989454 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.989419    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6958d802-bvzl8"]
Aug 04 12:08:45.990762 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:45.990737    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:46.113479 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.113457    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:46.116526 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.116505    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:46.194340 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194313    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69tvw\" (UniqueName: \"kubernetes.io/projected/a53b3b83-e00e-4630-9975-ce4728713836-kube-api-access-69tvw\") pod \"a53b3b83-e00e-4630-9975-ce4728713836\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") "
Aug 04 12:08:46.194491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194357    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-openshift-service-ca\") pod \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") "
Aug 04 12:08:46.194491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194386    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bspz\" (UniqueName: \"kubernetes.io/projected/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-kube-api-access-2bspz\") pod \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") "
Aug 04 12:08:46.194491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194419    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-maas-api-tls\") pod \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\" (UID: \"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf\") "
Aug 04 12:08:46.194491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194435    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a53b3b83-e00e-4630-9975-ce4728713836-plugins-config-volume\") pod \"a53b3b83-e00e-4630-9975-ce4728713836\" (UID: \"a53b3b83-e00e-4630-9975-ce4728713836\") "
Aug 04 12:08:46.194806 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194776    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" (UID: "b4fc735c-c631-4eef-9e7a-a9d14d8b7edf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:08:46.194938 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.194871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53b3b83-e00e-4630-9975-ce4728713836-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a53b3b83-e00e-4630-9975-ce4728713836" (UID: "a53b3b83-e00e-4630-9975-ce4728713836"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:08:46.196345 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.196315    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53b3b83-e00e-4630-9975-ce4728713836-kube-api-access-69tvw" (OuterVolumeSpecName: "kube-api-access-69tvw") pod "a53b3b83-e00e-4630-9975-ce4728713836" (UID: "a53b3b83-e00e-4630-9975-ce4728713836"). InnerVolumeSpecName "kube-api-access-69tvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:08:46.196453 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.196393    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-kube-api-access-2bspz" (OuterVolumeSpecName: "kube-api-access-2bspz") pod "b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" (UID: "b4fc735c-c631-4eef-9e7a-a9d14d8b7edf"). InnerVolumeSpecName "kube-api-access-2bspz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:08:46.196453 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.196417    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" (UID: "b4fc735c-c631-4eef-9e7a-a9d14d8b7edf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:08:46.295618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.295573    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:08:46.295618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.295612    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a53b3b83-e00e-4630-9975-ce4728713836-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:08:46.295618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.295622    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69tvw\" (UniqueName: \"kubernetes.io/projected/a53b3b83-e00e-4630-9975-ce4728713836-kube-api-access-69tvw\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:08:46.295618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.295633    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:08:46.295886 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.295641    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bspz\" (UniqueName: \"kubernetes.io/projected/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf-kube-api-access-2bspz\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:08:46.380753 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.380658    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:46.536842 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.536810    2558 generic.go:356] "Generic (PLEG): container finished" podID="b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" containerID="6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767" exitCode=0
Aug 04 12:08:46.537029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.536854    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"
Aug 04 12:08:46.537029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.536880    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r" event={"ID":"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf","Type":"ContainerDied","Data":"6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767"}
Aug 04 12:08:46.537029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.536922    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r" event={"ID":"b4fc735c-c631-4eef-9e7a-a9d14d8b7edf","Type":"ContainerDied","Data":"8680d8058b34a4efae4b1dbdc318330b741b09ff7f1c49da191f741a8b9bea98"}
Aug 04 12:08:46.537029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.536952    2558 scope.go:117] "RemoveContainer" containerID="6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767"
Aug 04 12:08:46.538222 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.538198    2558 generic.go:356] "Generic (PLEG): container finished" podID="a53b3b83-e00e-4630-9975-ce4728713836" containerID="92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595" exitCode=0
Aug 04 12:08:46.538309 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.538243    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"
Aug 04 12:08:46.538309 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.538242    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js" event={"ID":"a53b3b83-e00e-4630-9975-ce4728713836","Type":"ContainerDied","Data":"92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595"}
Aug 04 12:08:46.538400 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.538352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js" event={"ID":"a53b3b83-e00e-4630-9975-ce4728713836","Type":"ContainerDied","Data":"9c751aa660e1c16dda233465e7b14d63cf0de7be36dd26531fa5bf1b88937d78"}
Aug 04 12:08:46.552894 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.552877    2558 scope.go:117] "RemoveContainer" containerID="6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767"
Aug 04 12:08:46.553189 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:08:46.553163    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767\": container with ID starting with 6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767 not found: ID does not exist" containerID="6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767"
Aug 04 12:08:46.553297 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.553194    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767"} err="failed to get container status \"6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767\": rpc error: code = NotFound desc = could not find container \"6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767\": container with ID starting with 6e6a6f5c21acf12355d9d91d459143fd7300f1c2d25b5a34414d7727b806e767 not found: ID does not exist"
Aug 04 12:08:46.553297 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.553211    2558 scope.go:117] "RemoveContainer" containerID="92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595"
Aug 04 12:08:46.560898 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.560881    2558 scope.go:117] "RemoveContainer" containerID="92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595"
Aug 04 12:08:46.561144 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:08:46.561124    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595\": container with ID starting with 92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595 not found: ID does not exist" containerID="92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595"
Aug 04 12:08:46.561209 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.561152    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595"} err="failed to get container status \"92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595\": rpc error: code = NotFound desc = could not find container \"92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595\": container with ID starting with 92cc1497912cd541cfd39cfe8b2f940e55c0addf5821721e618926591907f595 not found: ID does not exist"
Aug 04 12:08:46.565496 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.565474    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"]
Aug 04 12:08:46.570772 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.570753    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js"]
Aug 04 12:08:46.582673 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.582653    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"]
Aug 04 12:08:46.585796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.585777    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6958d802-d79974ddd-nrh4r"]
Aug 04 12:08:46.993911 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.993879    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a24323c-71d9-4064-bda0-f0cda9c8eb32" path="/var/lib/kubelet/pods/4a24323c-71d9-4064-bda0-f0cda9c8eb32/volumes"
Aug 04 12:08:46.994313 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.994235    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a53b3b83-e00e-4630-9975-ce4728713836" path="/var/lib/kubelet/pods/a53b3b83-e00e-4630-9975-ce4728713836/volumes"
Aug 04 12:08:46.994600 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:46.994587    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" path="/var/lib/kubelet/pods/b4fc735c-c631-4eef-9e7a-a9d14d8b7edf/volumes"
Aug 04 12:08:52.242512 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:52.242476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:52.343490 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:52.343457    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:52.513931 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:52.513900    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:52.737984 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:52.737952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:52.886121 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:52.886022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:54.410460 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:54.410417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:54.512170 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:54.512137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:58.797811 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:58.797734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:58.900967 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:58.900936    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:58.997499 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:58.997472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:08:59.139406 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:08:59.139325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:09:04.811476 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:09:04.811432    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:09:04.911637 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:09:04.911598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:09:40.170973 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:09:40.170928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:10:22.863484 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:10:22.863443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:10:22.964382 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:10:22.964341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:07.190995 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:07.190964    2558 scope.go:117] "RemoveContainer" containerID="731c8b9257510f6e2c56b023cb0212c76cf21df04c3a740501ab759fc71d9459"
Aug 04 12:11:07.199333 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:07.199313    2558 scope.go:117] "RemoveContainer" containerID="31c4c15518360f43e2939f30bfa68a421c13bb50e28f35f835a268d0b9cfbbba"
Aug 04 12:11:07.507579 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:07.507551    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:09.312213 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:09.312180    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:11.041624 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:11.041580    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:11.121337 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:11.121297    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:12.108951 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.108913    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:12.194985 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.194952    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"]
Aug 04 12:11:12.195359 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195344    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a53b3b83-e00e-4630-9975-ce4728713836" containerName="payload-processing"
Aug 04 12:11:12.195430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195363    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53b3b83-e00e-4630-9975-ce4728713836" containerName="payload-processing"
Aug 04 12:11:12.195430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195375    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" containerName="maas-api"
Aug 04 12:11:12.195430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195381    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" containerName="maas-api"
Aug 04 12:11:12.195430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195389    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a24323c-71d9-4064-bda0-f0cda9c8eb32" containerName="revoke-keys"
Aug 04 12:11:12.195430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195395    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a24323c-71d9-4064-bda0-f0cda9c8eb32" containerName="revoke-keys"
Aug 04 12:11:12.195659 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195451    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a53b3b83-e00e-4630-9975-ce4728713836" containerName="payload-processing"
Aug 04 12:11:12.195659 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195458    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4fc735c-c631-4eef-9e7a-a9d14d8b7edf" containerName="maas-api"
Aug 04 12:11:12.195659 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.195468    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a24323c-71d9-4064-bda0-f0cda9c8eb32" containerName="revoke-keys"
Aug 04 12:11:12.198651 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.198627    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.202679 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.202657    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f3708437-serving-cert\""
Aug 04 12:11:12.216620 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.216598    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"]
Aug 04 12:11:12.275715 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.275682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x2bz\" (UniqueName: \"kubernetes.io/projected/a886cb7c-56ef-4df5-b881-b79e5a41a209-kube-api-access-9x2bz\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.275910 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.275733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a886cb7c-56ef-4df5-b881-b79e5a41a209-maas-api-tls\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.275910 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.275828    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a886cb7c-56ef-4df5-b881-b79e5a41a209-openshift-service-ca\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.356492 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.356453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:12.376429 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.376361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a886cb7c-56ef-4df5-b881-b79e5a41a209-maas-api-tls\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.376429 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.376418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a886cb7c-56ef-4df5-b881-b79e5a41a209-openshift-service-ca\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.376599 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.376481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x2bz\" (UniqueName: \"kubernetes.io/projected/a886cb7c-56ef-4df5-b881-b79e5a41a209-kube-api-access-9x2bz\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.377143 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.377125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a886cb7c-56ef-4df5-b881-b79e5a41a209-openshift-service-ca\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.378968 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.378939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a886cb7c-56ef-4df5-b881-b79e5a41a209-maas-api-tls\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.389888 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.389865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x2bz\" (UniqueName: \"kubernetes.io/projected/a886cb7c-56ef-4df5-b881-b79e5a41a209-kube-api-access-9x2bz\") pod \"maas-api-e2e-mt-f3708437-5b479886fb-266rz\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.445978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.445944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:12.512056 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.512023    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:12.655817 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.655786    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"]
Aug 04 12:11:12.657750 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:11:12.657714    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda886cb7c_56ef_4df5_b881_b79e5a41a209.slice/crio-69b293661e4560acc1e6328c8276ed1c56228ab59c23b9a788e84b378fc5b009 WatchSource:0}: Error finding container 69b293661e4560acc1e6328c8276ed1c56228ab59c23b9a788e84b378fc5b009: Status 404 returned error can't find the container with id 69b293661e4560acc1e6328c8276ed1c56228ab59c23b9a788e84b378fc5b009
Aug 04 12:11:12.663350 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.663327    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 12:11:12.921413 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:12.921337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:13.020373 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.020341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:13.050673 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.050638    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"]
Aug 04 12:11:13.055518 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.055486    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.061340 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.061312    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f3708437-dockercfg-rbcfm\""
Aug 04 12:11:13.062598 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.062578    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f3708437\""
Aug 04 12:11:13.067247 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.067225    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"]
Aug 04 12:11:13.184407 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.184327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac96a1f-3233-49b5-8660-f0fd77fc067e-plugins-config-volume\") pod \"payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.184407 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.184381    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fhjk\" (UniqueName: \"kubernetes.io/projected/9ac96a1f-3233-49b5-8660-f0fd77fc067e-kube-api-access-8fhjk\") pod \"payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.284935 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.284873    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac96a1f-3233-49b5-8660-f0fd77fc067e-plugins-config-volume\") pod \"payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.285013 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.284934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fhjk\" (UniqueName: \"kubernetes.io/projected/9ac96a1f-3233-49b5-8660-f0fd77fc067e-kube-api-access-8fhjk\") pod \"payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.285843 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.285818    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac96a1f-3233-49b5-8660-f0fd77fc067e-plugins-config-volume\") pod \"payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.291590 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.291559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz" event={"ID":"a886cb7c-56ef-4df5-b881-b79e5a41a209","Type":"ContainerStarted","Data":"69b293661e4560acc1e6328c8276ed1c56228ab59c23b9a788e84b378fc5b009"}
Aug 04 12:11:13.294140 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.294118    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fhjk\" (UniqueName: \"kubernetes.io/projected/9ac96a1f-3233-49b5-8660-f0fd77fc067e-kube-api-access-8fhjk\") pod \"payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.366985 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.366956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:13.497684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:13.497662    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"]
Aug 04 12:11:13.499368 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:11:13.499339    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac96a1f_3233_49b5_8660_f0fd77fc067e.slice/crio-f0f7f1edbd40d4fd35fd9609a5517d070594dc90aa4e8d18d1c1c2ff3a27d7a1 WatchSource:0}: Error finding container f0f7f1edbd40d4fd35fd9609a5517d070594dc90aa4e8d18d1c1c2ff3a27d7a1: Status 404 returned error can't find the container with id f0f7f1edbd40d4fd35fd9609a5517d070594dc90aa4e8d18d1c1c2ff3a27d7a1
Aug 04 12:11:14.298096 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.298064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz" event={"ID":"a886cb7c-56ef-4df5-b881-b79e5a41a209","Type":"ContainerStarted","Data":"2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2"}
Aug 04 12:11:14.298592 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.298127    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:14.299619 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.299596    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt" event={"ID":"9ac96a1f-3233-49b5-8660-f0fd77fc067e","Type":"ContainerStarted","Data":"867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d"}
Aug 04 12:11:14.299738 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.299643    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt" event={"ID":"9ac96a1f-3233-49b5-8660-f0fd77fc067e","Type":"ContainerStarted","Data":"f0f7f1edbd40d4fd35fd9609a5517d070594dc90aa4e8d18d1c1c2ff3a27d7a1"}
Aug 04 12:11:14.299738 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.299673    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:14.320034 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.319993    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz" podStartSLOduration=1.703075092 podStartE2EDuration="2.319976946s" totalImagesPullingTime="616.901854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:11:14.317006966 +0000 UTC m=+2287.887919796" watchObservedRunningTime="2026-08-04 12:11:14.319976946 +0000 UTC m=+2287.890889750"
Aug 04 12:11:14.335496 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:14.335454    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt" podStartSLOduration=1.335440057 podStartE2EDuration="1.335440057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:11:14.333722026 +0000 UTC m=+2287.904634830" watchObservedRunningTime="2026-08-04 12:11:14.335440057 +0000 UTC m=+2287.906352860"
Aug 04 12:11:20.309789 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:20.309760    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:11:20.461711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:20.461679    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:25.306459 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:25.306427    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:11:35.917826 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:35.917789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:11:58.795307 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:11:58.795213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:12:07.220703 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:07.220672    2558 scope.go:117] "RemoveContainer" containerID="8062baddf3382d71123f5e823302fc7eb55b19c9a3250ea12c6452caf4101fbc"
Aug 04 12:12:07.229134 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:07.229106    2558 scope.go:117] "RemoveContainer" containerID="a7a5709d1ea4a04ca2620554908faf412e757a2f6544eafde85b1403fb736a89"
Aug 04 12:12:51.474425 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.474388    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"]
Aug 04 12:12:51.478121 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.478102    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:51.481536 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.481508    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:12:51.482537 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.482511    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"]
Aug 04 12:12:51.635439 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.635400    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6151298a-6df2-41d4-ad93-c48ed800b70b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3708437-65ts6\" (UID: \"6151298a-6df2-41d4-ad93-c48ed800b70b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:51.736449 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.736357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6151298a-6df2-41d4-ad93-c48ed800b70b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3708437-65ts6\" (UID: \"6151298a-6df2-41d4-ad93-c48ed800b70b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:51.736951 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.736931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6151298a-6df2-41d4-ad93-c48ed800b70b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3708437-65ts6\" (UID: \"6151298a-6df2-41d4-ad93-c48ed800b70b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:51.790075 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.790039    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:51.928769 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:51.928733    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"]
Aug 04 12:12:51.929344 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:12:51.929314    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6151298a_6df2_41d4_ad93_c48ed800b70b.slice/crio-5537c8151e92efcedfc360ce8aece1d8866a32ab4b73ca893087c05464501e79 WatchSource:0}: Error finding container 5537c8151e92efcedfc360ce8aece1d8866a32ab4b73ca893087c05464501e79: Status 404 returned error can't find the container with id 5537c8151e92efcedfc360ce8aece1d8866a32ab4b73ca893087c05464501e79
Aug 04 12:12:52.786249 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:52.786161    2558 generic.go:356] "Generic (PLEG): container finished" podID="6151298a-6df2-41d4-ad93-c48ed800b70b" containerID="d9ffa18ea6454cbc767d7048e26fe9c3012d14440d0c43c1cf7a0623068f3c13" exitCode=0
Aug 04 12:12:52.786622 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:52.786247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6" event={"ID":"6151298a-6df2-41d4-ad93-c48ed800b70b","Type":"ContainerDied","Data":"d9ffa18ea6454cbc767d7048e26fe9c3012d14440d0c43c1cf7a0623068f3c13"}
Aug 04 12:12:52.786622 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:52.786317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6" event={"ID":"6151298a-6df2-41d4-ad93-c48ed800b70b","Type":"ContainerStarted","Data":"5537c8151e92efcedfc360ce8aece1d8866a32ab4b73ca893087c05464501e79"}
Aug 04 12:12:53.923669 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:53.923638    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:54.054831 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:54.054747    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6151298a-6df2-41d4-ad93-c48ed800b70b-maas-api-service-ca\") pod \"6151298a-6df2-41d4-ad93-c48ed800b70b\" (UID: \"6151298a-6df2-41d4-ad93-c48ed800b70b\") "
Aug 04 12:12:54.055160 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:54.055140    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6151298a-6df2-41d4-ad93-c48ed800b70b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6151298a-6df2-41d4-ad93-c48ed800b70b" (UID: "6151298a-6df2-41d4-ad93-c48ed800b70b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:12:54.155841 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:54.155802    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6151298a-6df2-41d4-ad93-c48ed800b70b-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:12:54.797710 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:54.797683    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"
Aug 04 12:12:54.797710 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:54.797685    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6" event={"ID":"6151298a-6df2-41d4-ad93-c48ed800b70b","Type":"ContainerDied","Data":"5537c8151e92efcedfc360ce8aece1d8866a32ab4b73ca893087c05464501e79"}
Aug 04 12:12:54.797921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:12:54.797768    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5537c8151e92efcedfc360ce8aece1d8866a32ab4b73ca893087c05464501e79"
Aug 04 12:13:06.906703 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:06.906676    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"]
Aug 04 12:13:06.907135 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:06.906874    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz" podUID="a886cb7c-56ef-4df5-b881-b79e5a41a209" containerName="maas-api" containerID="cri-o://2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2" gracePeriod=30
Aug 04 12:13:06.935679 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:06.935641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:06.966032 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:06.966005    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"]
Aug 04 12:13:06.966304 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:06.966238    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt" podUID="9ac96a1f-3233-49b5-8660-f0fd77fc067e" containerName="payload-processing" containerID="cri-o://867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d" gracePeriod=30
Aug 04 12:13:07.037353 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.037297    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:07.091589 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.091562    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:13:07.101452 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.101421    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:13:07.136565 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.136537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:07.171655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.171577    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"]
Aug 04 12:13:07.176772 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.176750    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3708437-65ts6"]
Aug 04 12:13:07.213583 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.213561    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:13:07.236398 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.236374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:07.244961 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.244937    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:13:07.374431 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374400    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a886cb7c-56ef-4df5-b881-b79e5a41a209-openshift-service-ca\") pod \"a886cb7c-56ef-4df5-b881-b79e5a41a209\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") "
Aug 04 12:13:07.374605 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374463    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x2bz\" (UniqueName: \"kubernetes.io/projected/a886cb7c-56ef-4df5-b881-b79e5a41a209-kube-api-access-9x2bz\") pod \"a886cb7c-56ef-4df5-b881-b79e5a41a209\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") "
Aug 04 12:13:07.374605 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374527    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a886cb7c-56ef-4df5-b881-b79e5a41a209-maas-api-tls\") pod \"a886cb7c-56ef-4df5-b881-b79e5a41a209\" (UID: \"a886cb7c-56ef-4df5-b881-b79e5a41a209\") "
Aug 04 12:13:07.374605 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac96a1f-3233-49b5-8660-f0fd77fc067e-plugins-config-volume\") pod \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") "
Aug 04 12:13:07.374605 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374591    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fhjk\" (UniqueName: \"kubernetes.io/projected/9ac96a1f-3233-49b5-8660-f0fd77fc067e-kube-api-access-8fhjk\") pod \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\" (UID: \"9ac96a1f-3233-49b5-8660-f0fd77fc067e\") "
Aug 04 12:13:07.374831 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374794    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a886cb7c-56ef-4df5-b881-b79e5a41a209-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a886cb7c-56ef-4df5-b881-b79e5a41a209" (UID: "a886cb7c-56ef-4df5-b881-b79e5a41a209"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:13:07.375019 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.374996    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac96a1f-3233-49b5-8660-f0fd77fc067e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ac96a1f-3233-49b5-8660-f0fd77fc067e" (UID: "9ac96a1f-3233-49b5-8660-f0fd77fc067e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:13:07.376614 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.376593    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a886cb7c-56ef-4df5-b881-b79e5a41a209-kube-api-access-9x2bz" (OuterVolumeSpecName: "kube-api-access-9x2bz") pod "a886cb7c-56ef-4df5-b881-b79e5a41a209" (UID: "a886cb7c-56ef-4df5-b881-b79e5a41a209"). InnerVolumeSpecName "kube-api-access-9x2bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:13:07.376699 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.376663    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac96a1f-3233-49b5-8660-f0fd77fc067e-kube-api-access-8fhjk" (OuterVolumeSpecName: "kube-api-access-8fhjk") pod "9ac96a1f-3233-49b5-8660-f0fd77fc067e" (UID: "9ac96a1f-3233-49b5-8660-f0fd77fc067e"). InnerVolumeSpecName "kube-api-access-8fhjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:13:07.376699 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.376664    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a886cb7c-56ef-4df5-b881-b79e5a41a209-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a886cb7c-56ef-4df5-b881-b79e5a41a209" (UID: "a886cb7c-56ef-4df5-b881-b79e5a41a209"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:13:07.475411 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.475338    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a886cb7c-56ef-4df5-b881-b79e5a41a209-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:13:07.475411 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.475364    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x2bz\" (UniqueName: \"kubernetes.io/projected/a886cb7c-56ef-4df5-b881-b79e5a41a209-kube-api-access-9x2bz\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:13:07.475411 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.475374    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a886cb7c-56ef-4df5-b881-b79e5a41a209-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:13:07.475411 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.475385    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac96a1f-3233-49b5-8660-f0fd77fc067e-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:13:07.475411 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.475393    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fhjk\" (UniqueName: \"kubernetes.io/projected/9ac96a1f-3233-49b5-8660-f0fd77fc067e-kube-api-access-8fhjk\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:13:07.866088 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.866052    2558 generic.go:356] "Generic (PLEG): container finished" podID="a886cb7c-56ef-4df5-b881-b79e5a41a209" containerID="2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2" exitCode=0
Aug 04 12:13:07.866320 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.866103    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"
Aug 04 12:13:07.866320 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.866143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz" event={"ID":"a886cb7c-56ef-4df5-b881-b79e5a41a209","Type":"ContainerDied","Data":"2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2"}
Aug 04 12:13:07.866320 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.866197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz" event={"ID":"a886cb7c-56ef-4df5-b881-b79e5a41a209","Type":"ContainerDied","Data":"69b293661e4560acc1e6328c8276ed1c56228ab59c23b9a788e84b378fc5b009"}
Aug 04 12:13:07.866320 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.866221    2558 scope.go:117] "RemoveContainer" containerID="2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2"
Aug 04 12:13:07.867706 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.867685    2558 generic.go:356] "Generic (PLEG): container finished" podID="9ac96a1f-3233-49b5-8660-f0fd77fc067e" containerID="867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d" exitCode=0
Aug 04 12:13:07.867810 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.867721    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt" event={"ID":"9ac96a1f-3233-49b5-8660-f0fd77fc067e","Type":"ContainerDied","Data":"867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d"}
Aug 04 12:13:07.867810 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.867733    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"
Aug 04 12:13:07.867810 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.867751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt" event={"ID":"9ac96a1f-3233-49b5-8660-f0fd77fc067e","Type":"ContainerDied","Data":"f0f7f1edbd40d4fd35fd9609a5517d070594dc90aa4e8d18d1c1c2ff3a27d7a1"}
Aug 04 12:13:07.878896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.877641    2558 scope.go:117] "RemoveContainer" containerID="2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2"
Aug 04 12:13:07.879793 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:13:07.879767    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2\": container with ID starting with 2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2 not found: ID does not exist" containerID="2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2"
Aug 04 12:13:07.879897 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.879801    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2"} err="failed to get container status \"2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2\": rpc error: code = NotFound desc = could not find container \"2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2\": container with ID starting with 2f5b28ee8646184bd8ab734e540e2caae22f7b7e4dee09242f81bb56e4df98f2 not found: ID does not exist"
Aug 04 12:13:07.879897 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.879820    2558 scope.go:117] "RemoveContainer" containerID="867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d"
Aug 04 12:13:07.887937 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.887919    2558 scope.go:117] "RemoveContainer" containerID="867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d"
Aug 04 12:13:07.888191 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:13:07.888169    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d\": container with ID starting with 867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d not found: ID does not exist" containerID="867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d"
Aug 04 12:13:07.888258 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.888199    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d"} err="failed to get container status \"867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d\": rpc error: code = NotFound desc = could not find container \"867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d\": container with ID starting with 867c3a343f66cf59239340e8131f12b3bb4dd56ec463805800de654a19938e5d not found: ID does not exist"
Aug 04 12:13:07.898492 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.898461    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"]
Aug 04 12:13:07.906267 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.906236    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f3708437-5b479886fb-266rz"]
Aug 04 12:13:07.920875 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.920841    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"]
Aug 04 12:13:07.922837 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:07.922815    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt"]
Aug 04 12:13:08.994359 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:08.994328    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6151298a-6df2-41d4-ad93-c48ed800b70b" path="/var/lib/kubelet/pods/6151298a-6df2-41d4-ad93-c48ed800b70b/volumes"
Aug 04 12:13:08.994851 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:08.994819    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ac96a1f-3233-49b5-8660-f0fd77fc067e" path="/var/lib/kubelet/pods/9ac96a1f-3233-49b5-8660-f0fd77fc067e/volumes"
Aug 04 12:13:08.995235 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:08.995215    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a886cb7c-56ef-4df5-b881-b79e5a41a209" path="/var/lib/kubelet/pods/a886cb7c-56ef-4df5-b881-b79e5a41a209/volumes"
Aug 04 12:13:10.418197 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:10.418167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:15.496498 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:15.496464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:28.391117 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:28.391038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:49.059336 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:49.059299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:51.754455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:51.754418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:52.521959 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:52.521928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:58.295660 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:58.295621    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:58.411845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:58.411807    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:58.620391 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:58.620311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:13:58.832465 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:13:58.832414    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:10.742767 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:10.742732    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:10.845035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:10.845007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:10.982598 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:10.982560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:11.079274 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:11.079249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:11.180195 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:11.180167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:12.488937 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:12.488910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:18.719372 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:18.719341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:18.795974 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:18.795939    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:19.425656 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:19.425623    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:26.418253 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:26.418221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:26.523418 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:26.523388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:26.627162 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:26.627122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:37.164728 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:37.164698    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:37.403978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:37.403937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:14:58.377251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:14:58.377175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:01.518588 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:01.518551    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:04.257501 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:04.257469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:04.360797 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:04.360760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:04.845187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:04.845151    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:04.966747 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:04.966711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:05.100838 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:05.100749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:07.281070 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:07.281041    2558 scope.go:117] "RemoveContainer" containerID="141da40da5a80ffe4fc9c8081108582a379ab85e21cc4b67f2ee95cae0623564"
Aug 04 12:15:08.215844 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:08.215806    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:08.316552 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:08.316510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:09.299439 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.299399    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"]
Aug 04 12:15:09.300225 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300198    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a886cb7c-56ef-4df5-b881-b79e5a41a209" containerName="maas-api"
Aug 04 12:15:09.300225 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300227    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a886cb7c-56ef-4df5-b881-b79e5a41a209" containerName="maas-api"
Aug 04 12:15:09.300420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300256    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac96a1f-3233-49b5-8660-f0fd77fc067e" containerName="payload-processing"
Aug 04 12:15:09.300420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300265    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac96a1f-3233-49b5-8660-f0fd77fc067e" containerName="payload-processing"
Aug 04 12:15:09.300420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300313    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6151298a-6df2-41d4-ad93-c48ed800b70b" containerName="revoke-keys"
Aug 04 12:15:09.300420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300323    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6151298a-6df2-41d4-ad93-c48ed800b70b" containerName="revoke-keys"
Aug 04 12:15:09.300612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300502    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a886cb7c-56ef-4df5-b881-b79e5a41a209" containerName="maas-api"
Aug 04 12:15:09.300612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300525    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac96a1f-3233-49b5-8660-f0fd77fc067e" containerName="payload-processing"
Aug 04 12:15:09.300612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.300534    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6151298a-6df2-41d4-ad93-c48ed800b70b" containerName="revoke-keys"
Aug 04 12:15:09.305193 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.305172    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.308564 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.308543    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-55901c-serving-cert\""
Aug 04 12:15:09.308863 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.308791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"]
Aug 04 12:15:09.330364 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.330337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:09.425263 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.425220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dst7n\" (UniqueName: \"kubernetes.io/projected/8482e912-611a-4787-8499-483a6971f985-kube-api-access-dst7n\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.425515 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.425432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8482e912-611a-4787-8499-483a6971f985-openshift-service-ca\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.425515 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.425482    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.526069 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.526033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.526244 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.526127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dst7n\" (UniqueName: \"kubernetes.io/projected/8482e912-611a-4787-8499-483a6971f985-kube-api-access-dst7n\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.526244 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.526188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8482e912-611a-4787-8499-483a6971f985-openshift-service-ca\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.526244 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:15:09.526206    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-serving-cert: secret "maas-api-e2e-inf-a-55901c-serving-cert" not found
Aug 04 12:15:09.526411 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:15:09.526334    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls podName:8482e912-611a-4787-8499-483a6971f985 nodeName:}" failed. No retries permitted until 2026-08-04 12:15:10.026315717 +0000 UTC m=+2523.597228501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls") pod "maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" (UID: "8482e912-611a-4787-8499-483a6971f985") : secret "maas-api-e2e-inf-a-55901c-serving-cert" not found
Aug 04 12:15:09.526834 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.526816    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8482e912-611a-4787-8499-483a6971f985-openshift-service-ca\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.535030 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.534999    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dst7n\" (UniqueName: \"kubernetes.io/projected/8482e912-611a-4787-8499-483a6971f985-kube-api-access-dst7n\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:09.692313 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.692197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:09.834480 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.834445    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:09.932591 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.932557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:09.947572 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.947498    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"]
Aug 04 12:15:09.951076 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.951061    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:09.954084 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.954066    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-55901c-dockercfg-kq8gx\""
Aug 04 12:15:09.954201 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.954065    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-55901c\""
Aug 04 12:15:09.961092 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.961068    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"]
Aug 04 12:15:09.986788 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.986759    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"]
Aug 04 12:15:09.990650 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.990631    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:09.999364 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:09.999343    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"]
Aug 04 12:15:10.031374 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.031344    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgzqn\" (UniqueName: \"kubernetes.io/projected/daa54299-0139-425b-a069-7d2239e46351-kube-api-access-zgzqn\") pod \"payload-processing-e2e-inf-a-55901c-596775948d-hp756\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") " pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.031533 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.031415    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa54299-0139-425b-a069-7d2239e46351-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-55901c-596775948d-hp756\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") " pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.031533 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.031488    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:10.031533 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.031515    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.031683 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.031541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28ppv\" (UniqueName: \"kubernetes.io/projected/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-kube-api-access-28ppv\") pod \"payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.034002 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.033978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls\") pod \"maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:10.132424 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.132385    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgzqn\" (UniqueName: \"kubernetes.io/projected/daa54299-0139-425b-a069-7d2239e46351-kube-api-access-zgzqn\") pod \"payload-processing-e2e-inf-a-55901c-596775948d-hp756\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") " pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.132630 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.132436    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa54299-0139-425b-a069-7d2239e46351-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-55901c-596775948d-hp756\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") " pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.132630 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.132575    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.132630 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.132605    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28ppv\" (UniqueName: \"kubernetes.io/projected/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-kube-api-access-28ppv\") pod \"payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.133236 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.133214    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa54299-0139-425b-a069-7d2239e46351-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-55901c-596775948d-hp756\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") " pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.133390 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.133368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.142222 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.142189    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgzqn\" (UniqueName: \"kubernetes.io/projected/daa54299-0139-425b-a069-7d2239e46351-kube-api-access-zgzqn\") pod \"payload-processing-e2e-inf-a-55901c-596775948d-hp756\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") " pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.142368 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.142192    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28ppv\" (UniqueName: \"kubernetes.io/projected/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-kube-api-access-28ppv\") pod \"payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.217867 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.217794    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:10.262392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.262352    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:10.300933 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.300896    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:10.372929 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.372801    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"]
Aug 04 12:15:10.372929 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:15:10.372844    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8482e912_611a_4787_8499_483a6971f985.slice/crio-1763e24718a2fe0c7714b477c5827ca1e6be2097583848aa836d0945296d16b2 WatchSource:0}: Error finding container 1763e24718a2fe0c7714b477c5827ca1e6be2097583848aa836d0945296d16b2: Status 404 returned error can't find the container with id 1763e24718a2fe0c7714b477c5827ca1e6be2097583848aa836d0945296d16b2
Aug 04 12:15:10.430309 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.430261    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"]
Aug 04 12:15:10.431909 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:15:10.431880    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493f13b5_5ddf_4155_bfb7_331e3b1f1a58.slice/crio-68543351a4f7d07b3ce0f8efb4dd5c1ea660ef3bb56ffa9cedc159c4804a3ae1 WatchSource:0}: Error finding container 68543351a4f7d07b3ce0f8efb4dd5c1ea660ef3bb56ffa9cedc159c4804a3ae1: Status 404 returned error can't find the container with id 68543351a4f7d07b3ce0f8efb4dd5c1ea660ef3bb56ffa9cedc159c4804a3ae1
Aug 04 12:15:10.458145 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.458121    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"]
Aug 04 12:15:10.467511 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:15:10.467485    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa54299_0139_425b_a069_7d2239e46351.slice/crio-f9ea21eb0039213936776ee5c7ce03c15ab6c3597e459494375094b2f74c2e61 WatchSource:0}: Error finding container f9ea21eb0039213936776ee5c7ce03c15ab6c3597e459494375094b2f74c2e61: Status 404 returned error can't find the container with id f9ea21eb0039213936776ee5c7ce03c15ab6c3597e459494375094b2f74c2e61
Aug 04 12:15:10.470517 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.470488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" event={"ID":"493f13b5-5ddf-4155-bfb7-331e3b1f1a58","Type":"ContainerStarted","Data":"68543351a4f7d07b3ce0f8efb4dd5c1ea660ef3bb56ffa9cedc159c4804a3ae1"}
Aug 04 12:15:10.471741 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:10.471720    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" event={"ID":"8482e912-611a-4787-8499-483a6971f985","Type":"ContainerStarted","Data":"1763e24718a2fe0c7714b477c5827ca1e6be2097583848aa836d0945296d16b2"}
Aug 04 12:15:11.479196 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.479158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" event={"ID":"493f13b5-5ddf-4155-bfb7-331e3b1f1a58","Type":"ContainerStarted","Data":"a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144"}
Aug 04 12:15:11.479643 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.479338    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:11.480704 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.480679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" event={"ID":"8482e912-611a-4787-8499-483a6971f985","Type":"ContainerStarted","Data":"8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6"}
Aug 04 12:15:11.480803 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.480730    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:11.481984 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.481957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" event={"ID":"daa54299-0139-425b-a069-7d2239e46351","Type":"ContainerStarted","Data":"9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc"}
Aug 04 12:15:11.482061 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.482005    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" event={"ID":"daa54299-0139-425b-a069-7d2239e46351","Type":"ContainerStarted","Data":"f9ea21eb0039213936776ee5c7ce03c15ab6c3597e459494375094b2f74c2e61"}
Aug 04 12:15:11.482061 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.482025    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:11.496827 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.496785    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" podStartSLOduration=2.496772431 podStartE2EDuration="2.496772431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:15:11.496138005 +0000 UTC m=+2525.067050835" watchObservedRunningTime="2026-08-04 12:15:11.496772431 +0000 UTC m=+2525.067685234"
Aug 04 12:15:11.515521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.515473    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" podStartSLOduration=1.901477339 podStartE2EDuration="2.515455138s" totalImagesPullingTime="613.977799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:15:11.514411228 +0000 UTC m=+2525.085324033" watchObservedRunningTime="2026-08-04 12:15:11.515455138 +0000 UTC m=+2525.086367943"
Aug 04 12:15:11.533107 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:11.533057    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" podStartSLOduration=2.533040131 podStartE2EDuration="2.533040131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:15:11.530428929 +0000 UTC m=+2525.101341735" watchObservedRunningTime="2026-08-04 12:15:11.533040131 +0000 UTC m=+2525.103952941"
Aug 04 12:15:17.491991 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:17.491959    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:15:22.488723 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:22.488687    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:15:22.489180 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:22.488774    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:15:33.254836 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:33.254803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:33.592146 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:33.592061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:33.903998 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:33.903909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:34.244569 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:34.244479    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:36.406786 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:36.406745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:36.510207 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:36.510174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:37.504228 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.504185    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"]
Aug 04 12:15:37.509807 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.509778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.513322 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.513294    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-31323b-serving-cert\""
Aug 04 12:15:37.514069 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.514042    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"]
Aug 04 12:15:37.547083 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.547052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:37.581225 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.581191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85459a09-eba6-42af-bbb9-a075bf7d68da-openshift-service-ca\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.581437 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.581314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.581437 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.581350    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5l7k\" (UniqueName: \"kubernetes.io/projected/85459a09-eba6-42af-bbb9-a075bf7d68da-kube-api-access-h5l7k\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.681938 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.681901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.681938 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.681939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5l7k\" (UniqueName: \"kubernetes.io/projected/85459a09-eba6-42af-bbb9-a075bf7d68da-kube-api-access-h5l7k\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.682176 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.681977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85459a09-eba6-42af-bbb9-a075bf7d68da-openshift-service-ca\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.682176 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:15:37.682043    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-serving-cert: secret "maas-api-e2e-inf-b-31323b-serving-cert" not found
Aug 04 12:15:37.682176 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:15:37.682128    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls podName:85459a09-eba6-42af-bbb9-a075bf7d68da nodeName:}" failed. No retries permitted until 2026-08-04 12:15:38.182109495 +0000 UTC m=+2551.753022298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls") pod "maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" (UID: "85459a09-eba6-42af-bbb9-a075bf7d68da") : secret "maas-api-e2e-inf-b-31323b-serving-cert" not found
Aug 04 12:15:37.682638 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.682622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85459a09-eba6-42af-bbb9-a075bf7d68da-openshift-service-ca\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.691337 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.691312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5l7k\" (UniqueName: \"kubernetes.io/projected/85459a09-eba6-42af-bbb9-a075bf7d68da-kube-api-access-h5l7k\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:37.993168 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:37.993084    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:38.125421 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.125392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:38.186658 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.186625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:38.189078 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.189059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls\") pod \"maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:38.220769 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.220742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:15:38.422676 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.422641    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:38.561251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.561217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"]
Aug 04 12:15:38.562608 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:15:38.562582    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85459a09_eba6_42af_bbb9_a075bf7d68da.slice/crio-3a017600e7dd46dd5f57053da02b5b280dee903f37d809fd94169ba86daee1e4 WatchSource:0}: Error finding container 3a017600e7dd46dd5f57053da02b5b280dee903f37d809fd94169ba86daee1e4: Status 404 returned error can't find the container with id 3a017600e7dd46dd5f57053da02b5b280dee903f37d809fd94169ba86daee1e4
Aug 04 12:15:38.621834 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:38.621804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" event={"ID":"85459a09-eba6-42af-bbb9-a075bf7d68da","Type":"ContainerStarted","Data":"3a017600e7dd46dd5f57053da02b5b280dee903f37d809fd94169ba86daee1e4"}
Aug 04 12:15:39.628853 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:39.628820    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" event={"ID":"85459a09-eba6-42af-bbb9-a075bf7d68da","Type":"ContainerStarted","Data":"7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1"}
Aug 04 12:15:39.629255 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:39.628940    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:15:39.649739 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:39.649688    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" podStartSLOduration=2.247968353 podStartE2EDuration="2.649674962s" totalImagesPullingTime="401.706609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:15:39.647408212 +0000 UTC m=+2553.218321017" watchObservedRunningTime="2026-08-04 12:15:39.649674962 +0000 UTC m=+2553.220587813"
Aug 04 12:15:45.638822 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:15:45.638793    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:16:01.429259 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:01.429212    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:16:04.605444 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.605403    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"]
Aug 04 12:16:04.609055 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.609033    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.612347 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.612328    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-55901c\"/\"openshift-service-ca.crt\""
Aug 04 12:16:04.613750 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.613731    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-55901c\"/\"default-dockercfg-8fcfk\""
Aug 04 12:16:04.613845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.613805    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-55901c\"/\"test-model-55901c-kserve-self-signed-certs\""
Aug 04 12:16:04.613845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.613834    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-55901c\"/\"kube-root-ca.crt\""
Aug 04 12:16:04.619051 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.619028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"]
Aug 04 12:16:04.651073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.651045    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:16:04.699522 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.699490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8v6n\" (UniqueName: \"kubernetes.io/projected/f867bd16-777c-4859-acce-26c80e2e8181-kube-api-access-k8v6n\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.699684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.699530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-dshm\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.699684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.699580    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f867bd16-777c-4859-acce-26c80e2e8181-tls-certs\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.699684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.699607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-home\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.699684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.699671    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-model-cache\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.699825 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.699705    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-tmp-dir\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.800667 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.800632    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8v6n\" (UniqueName: \"kubernetes.io/projected/f867bd16-777c-4859-acce-26c80e2e8181-kube-api-access-k8v6n\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.800868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.800674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-dshm\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.800868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.800789    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f867bd16-777c-4859-acce-26c80e2e8181-tls-certs\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.800868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.800820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-home\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.801048 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.800888    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-model-cache\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.801048 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.800922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-tmp-dir\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.801221 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.801190    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-model-cache\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.801329 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.801243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-home\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.801329 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.801274    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-tmp-dir\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.802971 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.802951    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-dshm\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.803308 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.803290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f867bd16-777c-4859-acce-26c80e2e8181-tls-certs\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.809545 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.809525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8v6n\" (UniqueName: \"kubernetes.io/projected/f867bd16-777c-4859-acce-26c80e2e8181-kube-api-access-k8v6n\") pod \"test-model-55901c-kserve-5846876499-m69st\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") " pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:04.919410 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:04.919317    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:05.065171 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:05.065144    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"]
Aug 04 12:16:05.066862 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:16:05.066833    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf867bd16_777c_4859_acce_26c80e2e8181.slice/crio-ce613681d7d34a04b9eb5b6db600cd2e524e32187708bc294af3a24c65a50c89 WatchSource:0}: Error finding container ce613681d7d34a04b9eb5b6db600cd2e524e32187708bc294af3a24c65a50c89: Status 404 returned error can't find the container with id ce613681d7d34a04b9eb5b6db600cd2e524e32187708bc294af3a24c65a50c89
Aug 04 12:16:05.779790 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:05.779754    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st" event={"ID":"f867bd16-777c-4859-acce-26c80e2e8181","Type":"ContainerStarted","Data":"00357d2d23d9fdc9e1d63d864eb67b53090240d1e703e889b21645a980c90e97"}
Aug 04 12:16:05.779790 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:05.779801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st" event={"ID":"f867bd16-777c-4859-acce-26c80e2e8181","Type":"ContainerStarted","Data":"ce613681d7d34a04b9eb5b6db600cd2e524e32187708bc294af3a24c65a50c89"}
Aug 04 12:16:05.799805 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:05.799738    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st" podStartSLOduration=1.7997213699999999 podStartE2EDuration="1.79972137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:16:05.798217435 +0000 UTC m=+2579.369130238" watchObservedRunningTime="2026-08-04 12:16:05.79972137 +0000 UTC m=+2579.370634174"
Aug 04 12:16:14.920394 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:14.920359    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:14.920845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:14.920414    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:14.932807 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:14.932785    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:15.843078 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:15.843046    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:16:22.747239 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:22.747204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:16:24.844580 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:16:24.844547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:00.880714 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:00.880681    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"]
Aug 04 12:17:00.903917 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:00.903880    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:01.855698 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:01.855662    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:12.358416 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.358383    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"]
Aug 04 12:17:12.358877 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.358666    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" podUID="8482e912-611a-4787-8499-483a6971f985" containerName="maas-api" containerID="cri-o://8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6" gracePeriod=30
Aug 04 12:17:12.385462 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.385437    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:12.412104 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.412074    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"]
Aug 04 12:17:12.412377 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.412331    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" podUID="daa54299-0139-425b-a069-7d2239e46351" containerName="payload-processing" containerID="cri-o://9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc" gracePeriod=30
Aug 04 12:17:12.418426 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.418402    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"]
Aug 04 12:17:12.418665 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.418631    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" podUID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" containerName="payload-pre-processing" containerID="cri-o://a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144" gracePeriod=30
Aug 04 12:17:12.484540 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.484512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:12.487602 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.487569    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" podUID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.69:9004: connect: connection refused"
Aug 04 12:17:12.487710 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.487567    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" podUID="daa54299-0139-425b-a069-7d2239e46351" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.70:9004: connect: connection refused"
Aug 04 12:17:12.487778 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.487574    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" podUID="8482e912-611a-4787-8499-483a6971f985" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.68:8443/health\": dial tcp 10.132.0.68:8443: connect: connection refused"
Aug 04 12:17:12.576586 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.576539    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:12.648831 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.648795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:12.667856 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.667829    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:17:12.709648 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.709619    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:17:12.713732 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.713709    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:17:12.791258 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791228    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa54299-0139-425b-a069-7d2239e46351-plugins-config-volume\") pod \"daa54299-0139-425b-a069-7d2239e46351\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") "
Aug 04 12:17:12.791447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791268    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgzqn\" (UniqueName: \"kubernetes.io/projected/daa54299-0139-425b-a069-7d2239e46351-kube-api-access-zgzqn\") pod \"daa54299-0139-425b-a069-7d2239e46351\" (UID: \"daa54299-0139-425b-a069-7d2239e46351\") "
Aug 04 12:17:12.791447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8482e912-611a-4787-8499-483a6971f985-openshift-service-ca\") pod \"8482e912-611a-4787-8499-483a6971f985\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") "
Aug 04 12:17:12.791447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791351    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls\") pod \"8482e912-611a-4787-8499-483a6971f985\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") "
Aug 04 12:17:12.791447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791372    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-plugins-config-volume\") pod \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") "
Aug 04 12:17:12.791447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791391    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28ppv\" (UniqueName: \"kubernetes.io/projected/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-kube-api-access-28ppv\") pod \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\" (UID: \"493f13b5-5ddf-4155-bfb7-331e3b1f1a58\") "
Aug 04 12:17:12.791734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791505    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dst7n\" (UniqueName: \"kubernetes.io/projected/8482e912-611a-4787-8499-483a6971f985-kube-api-access-dst7n\") pod \"8482e912-611a-4787-8499-483a6971f985\" (UID: \"8482e912-611a-4787-8499-483a6971f985\") "
Aug 04 12:17:12.791734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791682    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8482e912-611a-4787-8499-483a6971f985-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8482e912-611a-4787-8499-483a6971f985" (UID: "8482e912-611a-4787-8499-483a6971f985"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:17:12.791734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791701    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa54299-0139-425b-a069-7d2239e46351-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "daa54299-0139-425b-a069-7d2239e46351" (UID: "daa54299-0139-425b-a069-7d2239e46351"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:17:12.791734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791709    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "493f13b5-5ddf-4155-bfb7-331e3b1f1a58" (UID: "493f13b5-5ddf-4155-bfb7-331e3b1f1a58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:17:12.791957 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791817    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa54299-0139-425b-a069-7d2239e46351-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:12.791957 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791832    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8482e912-611a-4787-8499-483a6971f985-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:12.791957 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.791841    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:12.793554 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.793525    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8482e912-611a-4787-8499-483a6971f985-kube-api-access-dst7n" (OuterVolumeSpecName: "kube-api-access-dst7n") pod "8482e912-611a-4787-8499-483a6971f985" (UID: "8482e912-611a-4787-8499-483a6971f985"). InnerVolumeSpecName "kube-api-access-dst7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:17:12.793668 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.793613    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa54299-0139-425b-a069-7d2239e46351-kube-api-access-zgzqn" (OuterVolumeSpecName: "kube-api-access-zgzqn") pod "daa54299-0139-425b-a069-7d2239e46351" (UID: "daa54299-0139-425b-a069-7d2239e46351"). InnerVolumeSpecName "kube-api-access-zgzqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:17:12.793668 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.793617    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8482e912-611a-4787-8499-483a6971f985" (UID: "8482e912-611a-4787-8499-483a6971f985"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:17:12.793668 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.793643    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-kube-api-access-28ppv" (OuterVolumeSpecName: "kube-api-access-28ppv") pod "493f13b5-5ddf-4155-bfb7-331e3b1f1a58" (UID: "493f13b5-5ddf-4155-bfb7-331e3b1f1a58"). InnerVolumeSpecName "kube-api-access-28ppv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:17:12.893100 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.893012    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgzqn\" (UniqueName: \"kubernetes.io/projected/daa54299-0139-425b-a069-7d2239e46351-kube-api-access-zgzqn\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:12.893100 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.893055    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8482e912-611a-4787-8499-483a6971f985-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:12.893100 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.893071    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28ppv\" (UniqueName: \"kubernetes.io/projected/493f13b5-5ddf-4155-bfb7-331e3b1f1a58-kube-api-access-28ppv\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:12.893100 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:12.893087    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dst7n\" (UniqueName: \"kubernetes.io/projected/8482e912-611a-4787-8499-483a6971f985-kube-api-access-dst7n\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:13.001202 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.001167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:13.125343 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.125309    2558 generic.go:356] "Generic (PLEG): container finished" podID="daa54299-0139-425b-a069-7d2239e46351" containerID="9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc" exitCode=0
Aug 04 12:17:13.125497 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.125355    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"
Aug 04 12:17:13.125497 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.125384    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" event={"ID":"daa54299-0139-425b-a069-7d2239e46351","Type":"ContainerDied","Data":"9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc"}
Aug 04 12:17:13.125497 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.125431    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756" event={"ID":"daa54299-0139-425b-a069-7d2239e46351","Type":"ContainerDied","Data":"f9ea21eb0039213936776ee5c7ce03c15ab6c3597e459494375094b2f74c2e61"}
Aug 04 12:17:13.125497 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.125457    2558 scope.go:117] "RemoveContainer" containerID="9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc"
Aug 04 12:17:13.126677 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.126647    2558 generic.go:356] "Generic (PLEG): container finished" podID="8482e912-611a-4787-8499-483a6971f985" containerID="8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6" exitCode=0
Aug 04 12:17:13.126796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.126688    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"
Aug 04 12:17:13.126796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.126687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" event={"ID":"8482e912-611a-4787-8499-483a6971f985","Type":"ContainerDied","Data":"8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6"}
Aug 04 12:17:13.126796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.126735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6" event={"ID":"8482e912-611a-4787-8499-483a6971f985","Type":"ContainerDied","Data":"1763e24718a2fe0c7714b477c5827ca1e6be2097583848aa836d0945296d16b2"}
Aug 04 12:17:13.128220 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.128196    2558 generic.go:356] "Generic (PLEG): container finished" podID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" containerID="a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144" exitCode=0
Aug 04 12:17:13.128436 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.128225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" event={"ID":"493f13b5-5ddf-4155-bfb7-331e3b1f1a58","Type":"ContainerDied","Data":"a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144"}
Aug 04 12:17:13.128436 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.128238    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"
Aug 04 12:17:13.128436 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.128263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz" event={"ID":"493f13b5-5ddf-4155-bfb7-331e3b1f1a58","Type":"ContainerDied","Data":"68543351a4f7d07b3ce0f8efb4dd5c1ea660ef3bb56ffa9cedc159c4804a3ae1"}
Aug 04 12:17:13.135098 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.135084    2558 scope.go:117] "RemoveContainer" containerID="9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc"
Aug 04 12:17:13.135365 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:17:13.135342    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc\": container with ID starting with 9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc not found: ID does not exist" containerID="9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc"
Aug 04 12:17:13.135410 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.135378    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc"} err="failed to get container status \"9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc\": rpc error: code = NotFound desc = could not find container \"9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc\": container with ID starting with 9faaef1cd7caa96f04042db2f0c56ffe64eeede9a29a9c25b6e5cd69cd10e5bc not found: ID does not exist"
Aug 04 12:17:13.135410 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.135401    2558 scope.go:117] "RemoveContainer" containerID="8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6"
Aug 04 12:17:13.142770 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.142755    2558 scope.go:117] "RemoveContainer" containerID="8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6"
Aug 04 12:17:13.142999 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:17:13.142981    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6\": container with ID starting with 8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6 not found: ID does not exist" containerID="8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6"
Aug 04 12:17:13.143054 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.143004    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6"} err="failed to get container status \"8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6\": rpc error: code = NotFound desc = could not find container \"8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6\": container with ID starting with 8e9e3e1a004ef8f7c0fa8d33fbc7f35540c411355b33c3c7e0c43c63fdf878d6 not found: ID does not exist"
Aug 04 12:17:13.143054 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.143019    2558 scope.go:117] "RemoveContainer" containerID="a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144"
Aug 04 12:17:13.150105 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.150091    2558 scope.go:117] "RemoveContainer" containerID="a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144"
Aug 04 12:17:13.150367 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:17:13.150319    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144\": container with ID starting with a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144 not found: ID does not exist" containerID="a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144"
Aug 04 12:17:13.150367 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.150342    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144"} err="failed to get container status \"a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144\": rpc error: code = NotFound desc = could not find container \"a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144\": container with ID starting with a72a96f4b22206600f838c24893551816d378a1239a4fceaa4b67c1becf52144 not found: ID does not exist"
Aug 04 12:17:13.153589 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.153565    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"]
Aug 04 12:17:13.157683 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.157665    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz"]
Aug 04 12:17:13.171551 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.171532    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"]
Aug 04 12:17:13.175647 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.175623    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756"]
Aug 04 12:17:13.187773 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.187755    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"]
Aug 04 12:17:13.192001 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:13.191983    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-55901c-75f99bd78f-ssjf6"]
Aug 04 12:17:14.993980 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:14.993946    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" path="/var/lib/kubelet/pods/493f13b5-5ddf-4155-bfb7-331e3b1f1a58/volumes"
Aug 04 12:17:14.994389 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:14.994374    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8482e912-611a-4787-8499-483a6971f985" path="/var/lib/kubelet/pods/8482e912-611a-4787-8499-483a6971f985/volumes"
Aug 04 12:17:14.994700 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:14.994688    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daa54299-0139-425b-a069-7d2239e46351" path="/var/lib/kubelet/pods/daa54299-0139-425b-a069-7d2239e46351/volumes"
Aug 04 12:17:15.899951 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:15.899907    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st" podUID="f867bd16-777c-4859-acce-26c80e2e8181" containerName="main" containerID="cri-o://00357d2d23d9fdc9e1d63d864eb67b53090240d1e703e889b21645a980c90e97" gracePeriod=45
Aug 04 12:17:16.148113 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.148074    2558 generic.go:356] "Generic (PLEG): container finished" podID="f867bd16-777c-4859-acce-26c80e2e8181" containerID="00357d2d23d9fdc9e1d63d864eb67b53090240d1e703e889b21645a980c90e97" exitCode=0
Aug 04 12:17:16.148524 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.148131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st" event={"ID":"f867bd16-777c-4859-acce-26c80e2e8181","Type":"ContainerDied","Data":"00357d2d23d9fdc9e1d63d864eb67b53090240d1e703e889b21645a980c90e97"}
Aug 04 12:17:16.148524 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.148166    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st" event={"ID":"f867bd16-777c-4859-acce-26c80e2e8181","Type":"ContainerDied","Data":"ce613681d7d34a04b9eb5b6db600cd2e524e32187708bc294af3a24c65a50c89"}
Aug 04 12:17:16.148524 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.148182    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce613681d7d34a04b9eb5b6db600cd2e524e32187708bc294af3a24c65a50c89"
Aug 04 12:17:16.152218 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.152159    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:17:16.328023 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.327980    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-dshm\") pod \"f867bd16-777c-4859-acce-26c80e2e8181\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") "
Aug 04 12:17:16.328231 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-model-cache\") pod \"f867bd16-777c-4859-acce-26c80e2e8181\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") "
Aug 04 12:17:16.328231 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328073    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8v6n\" (UniqueName: \"kubernetes.io/projected/f867bd16-777c-4859-acce-26c80e2e8181-kube-api-access-k8v6n\") pod \"f867bd16-777c-4859-acce-26c80e2e8181\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") "
Aug 04 12:17:16.328231 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328129    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-home\") pod \"f867bd16-777c-4859-acce-26c80e2e8181\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") "
Aug 04 12:17:16.328231 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328158    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-tmp-dir\") pod \"f867bd16-777c-4859-acce-26c80e2e8181\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") "
Aug 04 12:17:16.328511 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328245    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f867bd16-777c-4859-acce-26c80e2e8181-tls-certs\") pod \"f867bd16-777c-4859-acce-26c80e2e8181\" (UID: \"f867bd16-777c-4859-acce-26c80e2e8181\") "
Aug 04 12:17:16.328511 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328424    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-model-cache" (OuterVolumeSpecName: "model-cache") pod "f867bd16-777c-4859-acce-26c80e2e8181" (UID: "f867bd16-777c-4859-acce-26c80e2e8181"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:17:16.328511 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328454    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-home" (OuterVolumeSpecName: "home") pod "f867bd16-777c-4859-acce-26c80e2e8181" (UID: "f867bd16-777c-4859-acce-26c80e2e8181"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:17:16.328636 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328595    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:16.328636 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328615    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:16.328636 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.328623    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f867bd16-777c-4859-acce-26c80e2e8181" (UID: "f867bd16-777c-4859-acce-26c80e2e8181"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:17:16.330566 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.330540    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f867bd16-777c-4859-acce-26c80e2e8181-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f867bd16-777c-4859-acce-26c80e2e8181" (UID: "f867bd16-777c-4859-acce-26c80e2e8181"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:17:16.330725 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.330708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-dshm" (OuterVolumeSpecName: "dshm") pod "f867bd16-777c-4859-acce-26c80e2e8181" (UID: "f867bd16-777c-4859-acce-26c80e2e8181"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:17:16.330774 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.330755    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f867bd16-777c-4859-acce-26c80e2e8181-kube-api-access-k8v6n" (OuterVolumeSpecName: "kube-api-access-k8v6n") pod "f867bd16-777c-4859-acce-26c80e2e8181" (UID: "f867bd16-777c-4859-acce-26c80e2e8181"). InnerVolumeSpecName "kube-api-access-k8v6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:17:16.429781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.429701    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:16.429781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.429730    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8v6n\" (UniqueName: \"kubernetes.io/projected/f867bd16-777c-4859-acce-26c80e2e8181-kube-api-access-k8v6n\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:16.429781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.429760    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f867bd16-777c-4859-acce-26c80e2e8181-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:16.429781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:16.429771    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f867bd16-777c-4859-acce-26c80e2e8181-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:17.153848 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:17.153817    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"
Aug 04 12:17:17.187041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:17.187017    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"]
Aug 04 12:17:17.197916 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:17.197894    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-55901c/test-model-55901c-kserve-5846876499-m69st"]
Aug 04 12:17:18.996382 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:18.996354    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f867bd16-777c-4859-acce-26c80e2e8181" path="/var/lib/kubelet/pods/f867bd16-777c-4859-acce-26c80e2e8181/volumes"
Aug 04 12:17:19.869649 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:19.869620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:20.035208 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:20.035177    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:20.223087 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:20.223003    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:20.450783 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:20.450752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:31.187542 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.187506    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"]
Aug 04 12:17:31.187975 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.187690    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" podUID="85459a09-eba6-42af-bbb9-a075bf7d68da" containerName="maas-api" containerID="cri-o://7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1" gracePeriod=30
Aug 04 12:17:31.231655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.231622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:31.346862 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.346827    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:31.439922 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.439857    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:17:31.453355 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.453334    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85459a09-eba6-42af-bbb9-a075bf7d68da-openshift-service-ca\") pod \"85459a09-eba6-42af-bbb9-a075bf7d68da\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") "
Aug 04 12:17:31.453461 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.453435    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5l7k\" (UniqueName: \"kubernetes.io/projected/85459a09-eba6-42af-bbb9-a075bf7d68da-kube-api-access-h5l7k\") pod \"85459a09-eba6-42af-bbb9-a075bf7d68da\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") "
Aug 04 12:17:31.453511 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.453475    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls\") pod \"85459a09-eba6-42af-bbb9-a075bf7d68da\" (UID: \"85459a09-eba6-42af-bbb9-a075bf7d68da\") "
Aug 04 12:17:31.453678 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.453655    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85459a09-eba6-42af-bbb9-a075bf7d68da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85459a09-eba6-42af-bbb9-a075bf7d68da" (UID: "85459a09-eba6-42af-bbb9-a075bf7d68da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:17:31.455514 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.455489    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85459a09-eba6-42af-bbb9-a075bf7d68da-kube-api-access-h5l7k" (OuterVolumeSpecName: "kube-api-access-h5l7k") pod "85459a09-eba6-42af-bbb9-a075bf7d68da" (UID: "85459a09-eba6-42af-bbb9-a075bf7d68da"). InnerVolumeSpecName "kube-api-access-h5l7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:17:31.455589 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.455517    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85459a09-eba6-42af-bbb9-a075bf7d68da" (UID: "85459a09-eba6-42af-bbb9-a075bf7d68da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:17:31.495648 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.495620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:31.554981 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.554948    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5l7k\" (UniqueName: \"kubernetes.io/projected/85459a09-eba6-42af-bbb9-a075bf7d68da-kube-api-access-h5l7k\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:31.554981 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.554976    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85459a09-eba6-42af-bbb9-a075bf7d68da-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:31.555149 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.554990    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85459a09-eba6-42af-bbb9-a075bf7d68da-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:17:31.582409 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.582382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:31.955203 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:31.955169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:32.230142 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.230056    2558 generic.go:356] "Generic (PLEG): container finished" podID="85459a09-eba6-42af-bbb9-a075bf7d68da" containerID="7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1" exitCode=0
Aug 04 12:17:32.230142 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.230098    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"
Aug 04 12:17:32.230142 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.230137    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" event={"ID":"85459a09-eba6-42af-bbb9-a075bf7d68da","Type":"ContainerDied","Data":"7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1"}
Aug 04 12:17:32.230652 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.230185    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p" event={"ID":"85459a09-eba6-42af-bbb9-a075bf7d68da","Type":"ContainerDied","Data":"3a017600e7dd46dd5f57053da02b5b280dee903f37d809fd94169ba86daee1e4"}
Aug 04 12:17:32.230652 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.230207    2558 scope.go:117] "RemoveContainer" containerID="7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1"
Aug 04 12:17:32.239333 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.239316    2558 scope.go:117] "RemoveContainer" containerID="7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1"
Aug 04 12:17:32.239612 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:17:32.239595    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1\": container with ID starting with 7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1 not found: ID does not exist" containerID="7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1"
Aug 04 12:17:32.239658 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.239619    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1"} err="failed to get container status \"7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1\": rpc error: code = NotFound desc = could not find container \"7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1\": container with ID starting with 7d8239a730e9f5243e36deddee1592c205dc618938c9166469e135d5c31e78c1 not found: ID does not exist"
Aug 04 12:17:32.255049 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.255024    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"]
Aug 04 12:17:32.259796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.259778    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31323b-656c79d9c-vlc5p"]
Aug 04 12:17:32.993956 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:32.993925    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85459a09-eba6-42af-bbb9-a075bf7d68da" path="/var/lib/kubelet/pods/85459a09-eba6-42af-bbb9-a075bf7d68da/volumes"
Aug 04 12:17:40.132421 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:40.132388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:40.301509 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:40.301476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:40.438097 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:40.438025    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:40.804588 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:40.804557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:41.140454 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:41.140372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:44.393145 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:44.393110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:44.509393 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:44.509360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:45.559866 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.559834    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"]
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560183    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" containerName="payload-pre-processing"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560194    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" containerName="payload-pre-processing"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560209    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa54299-0139-425b-a069-7d2239e46351" containerName="payload-processing"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560216    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa54299-0139-425b-a069-7d2239e46351" containerName="payload-processing"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560228    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f867bd16-777c-4859-acce-26c80e2e8181" containerName="main"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560234    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f867bd16-777c-4859-acce-26c80e2e8181" containerName="main"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560243    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8482e912-611a-4787-8499-483a6971f985" containerName="maas-api"
Aug 04 12:17:45.560252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560248    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8482e912-611a-4787-8499-483a6971f985" containerName="maas-api"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560260    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85459a09-eba6-42af-bbb9-a075bf7d68da" containerName="maas-api"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560265    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="85459a09-eba6-42af-bbb9-a075bf7d68da" containerName="maas-api"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560353    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8482e912-611a-4787-8499-483a6971f985" containerName="maas-api"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560364    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="85459a09-eba6-42af-bbb9-a075bf7d68da" containerName="maas-api"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560370    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa54299-0139-425b-a069-7d2239e46351" containerName="payload-processing"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560377    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="493f13b5-5ddf-4155-bfb7-331e3b1f1a58" containerName="payload-pre-processing"
Aug 04 12:17:45.560521 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.560382    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f867bd16-777c-4859-acce-26c80e2e8181" containerName="main"
Aug 04 12:17:45.563536 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.563518    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.567498 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.567478    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-62afff-serving-cert\""
Aug 04 12:17:45.592918 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.592897    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"]
Aug 04 12:17:45.677817 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.677788    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:45.681990 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.681969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f826736-e414-4361-8d54-e92088c0deaf-openshift-service-ca\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.682072 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.682011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwh6z\" (UniqueName: \"kubernetes.io/projected/4f826736-e414-4361-8d54-e92088c0deaf-kube-api-access-cwh6z\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.682147 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.682129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.782881 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.782844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwh6z\" (UniqueName: \"kubernetes.io/projected/4f826736-e414-4361-8d54-e92088c0deaf-kube-api-access-cwh6z\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.783073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.782971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.783073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.783038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f826736-e414-4361-8d54-e92088c0deaf-openshift-service-ca\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.783190 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:17:45.783142    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-serving-cert: secret "maas-api-e2e-api-a-62afff-serving-cert" not found
Aug 04 12:17:45.783261 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:17:45.783249    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls podName:4f826736-e414-4361-8d54-e92088c0deaf nodeName:}" failed. No retries permitted until 2026-08-04 12:17:46.283229972 +0000 UTC m=+2679.854142759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls") pod "maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" (UID: "4f826736-e414-4361-8d54-e92088c0deaf") : secret "maas-api-e2e-api-a-62afff-serving-cert" not found
Aug 04 12:17:45.783693 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.783672    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f826736-e414-4361-8d54-e92088c0deaf-openshift-service-ca\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.811168 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.811104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwh6z\" (UniqueName: \"kubernetes.io/projected/4f826736-e414-4361-8d54-e92088c0deaf-kube-api-access-cwh6z\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:45.968663 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:45.968627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:46.289232 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.289190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:46.291747 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.291719    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls\") pod \"maas-api-e2e-api-a-62afff-65fdcf649b-9t85l\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:46.473240 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.473195    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:46.556068 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.555989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:46.629783 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.629760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"]
Aug 04 12:17:46.632270 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:17:46.632241    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f826736_e414_4361_8d54_e92088c0deaf.slice/crio-ac6450da77bc46b16477c97d4c4e8a642e9edcf8c0fdfe16260d881c188fa6db WatchSource:0}: Error finding container ac6450da77bc46b16477c97d4c4e8a642e9edcf8c0fdfe16260d881c188fa6db: Status 404 returned error can't find the container with id ac6450da77bc46b16477c97d4c4e8a642e9edcf8c0fdfe16260d881c188fa6db
Aug 04 12:17:46.633567 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.633530    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 12:17:46.744993 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.744963    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"]
Aug 04 12:17:46.749383 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.749367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:46.752024 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.751997    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-62afff\""
Aug 04 12:17:46.786032 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.786005    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"]
Aug 04 12:17:46.792546 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.792522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:17:46.793707 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.793679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3194e8e0-f08e-4475-aaeb-a416d5c64592-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:46.793807 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.793718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzfx\" (UniqueName: \"kubernetes.io/projected/3194e8e0-f08e-4475-aaeb-a416d5c64592-kube-api-access-dqzfx\") pod \"payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:46.895134 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.895036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3194e8e0-f08e-4475-aaeb-a416d5c64592-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:46.895134 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.895079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzfx\" (UniqueName: \"kubernetes.io/projected/3194e8e0-f08e-4475-aaeb-a416d5c64592-kube-api-access-dqzfx\") pod \"payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:46.895777 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.895755    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3194e8e0-f08e-4475-aaeb-a416d5c64592-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:46.908407 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:46.908387    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzfx\" (UniqueName: \"kubernetes.io/projected/3194e8e0-f08e-4475-aaeb-a416d5c64592-kube-api-access-dqzfx\") pod \"payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:47.059531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.059497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:47.248498 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.248459    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"]
Aug 04 12:17:47.252471 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:17:47.252431    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3194e8e0_f08e_4475_aaeb_a416d5c64592.slice/crio-06cd1693f7a6a68df83f1ff1843904319f5b34e90cf93b8f4455809bb1acded7 WatchSource:0}: Error finding container 06cd1693f7a6a68df83f1ff1843904319f5b34e90cf93b8f4455809bb1acded7: Status 404 returned error can't find the container with id 06cd1693f7a6a68df83f1ff1843904319f5b34e90cf93b8f4455809bb1acded7
Aug 04 12:17:47.304922 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.304895    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6" event={"ID":"3194e8e0-f08e-4475-aaeb-a416d5c64592","Type":"ContainerStarted","Data":"06cd1693f7a6a68df83f1ff1843904319f5b34e90cf93b8f4455809bb1acded7"}
Aug 04 12:17:47.306229 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.306207    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" event={"ID":"4f826736-e414-4361-8d54-e92088c0deaf","Type":"ContainerStarted","Data":"1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe"}
Aug 04 12:17:47.306368 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.306248    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" event={"ID":"4f826736-e414-4361-8d54-e92088c0deaf","Type":"ContainerStarted","Data":"ac6450da77bc46b16477c97d4c4e8a642e9edcf8c0fdfe16260d881c188fa6db"}
Aug 04 12:17:47.306368 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.306306    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:17:47.354045 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:47.353993    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" podStartSLOduration=1.883640152 podStartE2EDuration="2.353974808s" totalImagesPullingTime="470.334656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:17:47.353449978 +0000 UTC m=+2680.924362794" watchObservedRunningTime="2026-08-04 12:17:47.353974808 +0000 UTC m=+2680.924887613"
Aug 04 12:17:48.312894 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:48.312860    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6" event={"ID":"3194e8e0-f08e-4475-aaeb-a416d5c64592","Type":"ContainerStarted","Data":"01ba689cc9c28f93118fa3eeaf907cc08aa0418545e9d96b61230cf7ef6626e5"}
Aug 04 12:17:48.360226 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:48.360169    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6" podStartSLOduration=2.360150857 podStartE2EDuration="2.360150857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:17:48.359321039 +0000 UTC m=+2681.930233843" watchObservedRunningTime="2026-08-04 12:17:48.360150857 +0000 UTC m=+2681.931063661"
Aug 04 12:17:49.319577 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:49.319538    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:17:53.319950 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:17:53.319922    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:18:00.327274 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:00.327201    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:18:07.129249 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:07.129222    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:18:07.137680 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:07.137660    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:18:09.360468 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:09.360419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:09.997806 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:09.997776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:10.378586 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:10.378552    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:10.606541 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:10.606513    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:12.320685 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:12.320652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:12.420091 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:12.420062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:13.415463 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:13.415434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:13.509209 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:13.509182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:14.700535 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.700501    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"]
Aug 04 12:18:14.704327 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.704307    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.707142 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.707113    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9e17b5-serving-cert\""
Aug 04 12:18:14.721162 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.721136    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"]
Aug 04 12:18:14.740523 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.740496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:14.838734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.838705    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5933ad39-2f47-4ff1-81c1-957fc7bef788-openshift-service-ca\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.838896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.838748    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ksln\" (UniqueName: \"kubernetes.io/projected/5933ad39-2f47-4ff1-81c1-957fc7bef788-kube-api-access-2ksln\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.838896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.838833    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.939755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.939713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5933ad39-2f47-4ff1-81c1-957fc7bef788-openshift-service-ca\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.939755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.939763    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ksln\" (UniqueName: \"kubernetes.io/projected/5933ad39-2f47-4ff1-81c1-957fc7bef788-kube-api-access-2ksln\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.939969 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.939825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.939969 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:18:14.939955    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-serving-cert: secret "maas-api-e2e-api-b-9e17b5-serving-cert" not found
Aug 04 12:18:14.940045 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:18:14.940025    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls podName:5933ad39-2f47-4ff1-81c1-957fc7bef788 nodeName:}" failed. No retries permitted until 2026-08-04 12:18:15.440005734 +0000 UTC m=+2709.010918517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls") pod "maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" (UID: "5933ad39-2f47-4ff1-81c1-957fc7bef788") : secret "maas-api-e2e-api-b-9e17b5-serving-cert" not found
Aug 04 12:18:14.940433 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.940412    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5933ad39-2f47-4ff1-81c1-957fc7bef788-openshift-service-ca\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:14.955367 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:14.955293    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ksln\" (UniqueName: \"kubernetes.io/projected/5933ad39-2f47-4ff1-81c1-957fc7bef788-kube-api-access-2ksln\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:15.156910 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.156881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:15.360111 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.360084    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:15.444670 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.444638    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:15.447020 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.447000    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls\") pod \"maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:15.558816 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.558780    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"]
Aug 04 12:18:15.562348 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.562332    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.566626 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.566595    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9e17b5\""
Aug 04 12:18:15.566756 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.566708    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9e17b5-dockercfg-xw94g\""
Aug 04 12:18:15.571134 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.571110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:15.618664 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.618589    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:15.631813 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.631778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"]
Aug 04 12:18:15.665868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.665831    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"]
Aug 04 12:18:15.670455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.670431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.695197 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.695133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"]
Aug 04 12:18:15.747489 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.747455    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhsl5\" (UniqueName: \"kubernetes.io/projected/8e1b9860-6cef-41d2-a2a1-553d80d35291-kube-api-access-vhsl5\") pod \"payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.747917 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.747597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1b9860-6cef-41d2-a2a1-553d80d35291-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.770772 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.770742    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"]
Aug 04 12:18:15.772171 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:18:15.772139    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5933ad39_2f47_4ff1_81c1_957fc7bef788.slice/crio-1256a0e20885adf44149cc156eff34e96b6468ab1e84f8782ae47145b63fb8a7 WatchSource:0}: Error finding container 1256a0e20885adf44149cc156eff34e96b6468ab1e84f8782ae47145b63fb8a7: Status 404 returned error can't find the container with id 1256a0e20885adf44149cc156eff34e96b6468ab1e84f8782ae47145b63fb8a7
Aug 04 12:18:15.849050 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.849017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1b9860-6cef-41d2-a2a1-553d80d35291-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.849237 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.849063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86ac599-1735-4098-8886-c9faf01bae50-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") " pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.849237 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.849129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhsl5\" (UniqueName: \"kubernetes.io/projected/8e1b9860-6cef-41d2-a2a1-553d80d35291-kube-api-access-vhsl5\") pod \"payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.849237 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.849202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xcmj\" (UniqueName: \"kubernetes.io/projected/e86ac599-1735-4098-8886-c9faf01bae50-kube-api-access-9xcmj\") pod \"payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") " pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.849853 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.849831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1b9860-6cef-41d2-a2a1-553d80d35291-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.858652 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.858630    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhsl5\" (UniqueName: \"kubernetes.io/projected/8e1b9860-6cef-41d2-a2a1-553d80d35291-kube-api-access-vhsl5\") pod \"payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.873315 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.873240    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:15.950055 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.950024    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xcmj\" (UniqueName: \"kubernetes.io/projected/e86ac599-1735-4098-8886-c9faf01bae50-kube-api-access-9xcmj\") pod \"payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") " pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.950206 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.950080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86ac599-1735-4098-8886-c9faf01bae50-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") " pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.950806 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.950787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86ac599-1735-4098-8886-c9faf01bae50-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") " pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.962200 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.962144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xcmj\" (UniqueName: \"kubernetes.io/projected/e86ac599-1735-4098-8886-c9faf01bae50-kube-api-access-9xcmj\") pod \"payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") " pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:15.983880 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:15.983784    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:16.132436 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.132415    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"]
Aug 04 12:18:16.136391 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:18:16.136363    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86ac599_1735_4098_8886_c9faf01bae50.slice/crio-15068a3e8711ef9aa67dd4c8c4c461794c08f44bf6756c4bebb44dbaade73175 WatchSource:0}: Error finding container 15068a3e8711ef9aa67dd4c8c4c461794c08f44bf6756c4bebb44dbaade73175: Status 404 returned error can't find the container with id 15068a3e8711ef9aa67dd4c8c4c461794c08f44bf6756c4bebb44dbaade73175
Aug 04 12:18:16.221461 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.221434    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"]
Aug 04 12:18:16.223803 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:18:16.223773    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1b9860_6cef_41d2_a2a1_553d80d35291.slice/crio-0d828040d060cc16763878ed9117663be7ec526fb01876318ea7295924725803 WatchSource:0}: Error finding container 0d828040d060cc16763878ed9117663be7ec526fb01876318ea7295924725803: Status 404 returned error can't find the container with id 0d828040d060cc16763878ed9117663be7ec526fb01876318ea7295924725803
Aug 04 12:18:16.463856 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.463758    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" event={"ID":"5933ad39-2f47-4ff1-81c1-957fc7bef788","Type":"ContainerStarted","Data":"9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6"}
Aug 04 12:18:16.464015 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.463908    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:16.464015 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.463931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" event={"ID":"5933ad39-2f47-4ff1-81c1-957fc7bef788","Type":"ContainerStarted","Data":"1256a0e20885adf44149cc156eff34e96b6468ab1e84f8782ae47145b63fb8a7"}
Aug 04 12:18:16.465238 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.465213    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x" event={"ID":"e86ac599-1735-4098-8886-c9faf01bae50","Type":"ContainerStarted","Data":"94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323"}
Aug 04 12:18:16.465398 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.465259    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x" event={"ID":"e86ac599-1735-4098-8886-c9faf01bae50","Type":"ContainerStarted","Data":"15068a3e8711ef9aa67dd4c8c4c461794c08f44bf6756c4bebb44dbaade73175"}
Aug 04 12:18:16.465398 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.465300    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:16.466666 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.466641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn" event={"ID":"8e1b9860-6cef-41d2-a2a1-553d80d35291","Type":"ContainerStarted","Data":"95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7"}
Aug 04 12:18:16.466778 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.466680    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn" event={"ID":"8e1b9860-6cef-41d2-a2a1-553d80d35291","Type":"ContainerStarted","Data":"0d828040d060cc16763878ed9117663be7ec526fb01876318ea7295924725803"}
Aug 04 12:18:16.466778 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.466773    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:16.486941 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.486887    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" podStartSLOduration=2.134822091 podStartE2EDuration="2.486875049s" totalImagesPullingTime="352.052958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:18:16.48583963 +0000 UTC m=+2710.056752433" watchObservedRunningTime="2026-08-04 12:18:16.486875049 +0000 UTC m=+2710.057787870"
Aug 04 12:18:16.509479 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.509418    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn" podStartSLOduration=1.509402339 podStartE2EDuration="1.509402339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:18:16.508751623 +0000 UTC m=+2710.079664427" watchObservedRunningTime="2026-08-04 12:18:16.509402339 +0000 UTC m=+2710.080315142"
Aug 04 12:18:16.531942 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:16.531890    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x" podStartSLOduration=1.5318752629999999 podStartE2EDuration="1.531875263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:18:16.530635291 +0000 UTC m=+2710.101548093" watchObservedRunningTime="2026-08-04 12:18:16.531875263 +0000 UTC m=+2710.102788066"
Aug 04 12:18:22.477696 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:22.477666    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:18:27.474050 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:27.474018    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:18:27.474465 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:27.474102    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:18:38.499334 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:38.499299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:57.396822 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.396789    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"]
Aug 04 12:18:57.399180 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.397027    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" podUID="4f826736-e414-4361-8d54-e92088c0deaf" containerName="maas-api" containerID="cri-o://1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe" gracePeriod=30
Aug 04 12:18:57.450335 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.450303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:57.547888 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.547854    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"]
Aug 04 12:18:57.548170 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.548146    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6" podUID="3194e8e0-f08e-4475-aaeb-a416d5c64592" containerName="payload-pre-processing" containerID="cri-o://01ba689cc9c28f93118fa3eeaf907cc08aa0418545e9d96b61230cf7ef6626e5" gracePeriod=30
Aug 04 12:18:57.676262 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.676233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:57.677609 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.677570    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:18:57.680185 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.680160    2558 generic.go:356] "Generic (PLEG): container finished" podID="3194e8e0-f08e-4475-aaeb-a416d5c64592" containerID="01ba689cc9c28f93118fa3eeaf907cc08aa0418545e9d96b61230cf7ef6626e5" exitCode=0
Aug 04 12:18:57.680333 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.680204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6" event={"ID":"3194e8e0-f08e-4475-aaeb-a416d5c64592","Type":"ContainerDied","Data":"01ba689cc9c28f93118fa3eeaf907cc08aa0418545e9d96b61230cf7ef6626e5"}
Aug 04 12:18:57.681429 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.681407    2558 generic.go:356] "Generic (PLEG): container finished" podID="4f826736-e414-4361-8d54-e92088c0deaf" containerID="1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe" exitCode=0
Aug 04 12:18:57.681559 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.681456    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"
Aug 04 12:18:57.681559 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.681505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" event={"ID":"4f826736-e414-4361-8d54-e92088c0deaf","Type":"ContainerDied","Data":"1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe"}
Aug 04 12:18:57.681559 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.681554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l" event={"ID":"4f826736-e414-4361-8d54-e92088c0deaf","Type":"ContainerDied","Data":"ac6450da77bc46b16477c97d4c4e8a642e9edcf8c0fdfe16260d881c188fa6db"}
Aug 04 12:18:57.681730 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.681584    2558 scope.go:117] "RemoveContainer" containerID="1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe"
Aug 04 12:18:57.691450 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.691434    2558 scope.go:117] "RemoveContainer" containerID="1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe"
Aug 04 12:18:57.693509 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:18:57.693479    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe\": container with ID starting with 1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe not found: ID does not exist" containerID="1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe"
Aug 04 12:18:57.693622 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.693520    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe"} err="failed to get container status \"1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe\": rpc error: code = NotFound desc = could not find container \"1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe\": container with ID starting with 1a69184d5f5275b3f2b12ed5b224785c25918fbe39545b5f17323b9304936dfe not found: ID does not exist"
Aug 04 12:18:57.810865 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.810839    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:18:57.822164 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.822135    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:57.841147 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.841119    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f826736-e414-4361-8d54-e92088c0deaf-openshift-service-ca\") pod \"4f826736-e414-4361-8d54-e92088c0deaf\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") "
Aug 04 12:18:57.841296 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.841197    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls\") pod \"4f826736-e414-4361-8d54-e92088c0deaf\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") "
Aug 04 12:18:57.841349 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.841309    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwh6z\" (UniqueName: \"kubernetes.io/projected/4f826736-e414-4361-8d54-e92088c0deaf-kube-api-access-cwh6z\") pod \"4f826736-e414-4361-8d54-e92088c0deaf\" (UID: \"4f826736-e414-4361-8d54-e92088c0deaf\") "
Aug 04 12:18:57.841471 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.841448    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f826736-e414-4361-8d54-e92088c0deaf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f826736-e414-4361-8d54-e92088c0deaf" (UID: "4f826736-e414-4361-8d54-e92088c0deaf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:18:57.841673 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.841652    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f826736-e414-4361-8d54-e92088c0deaf-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:18:57.843226 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.843201    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f826736-e414-4361-8d54-e92088c0deaf-kube-api-access-cwh6z" (OuterVolumeSpecName: "kube-api-access-cwh6z") pod "4f826736-e414-4361-8d54-e92088c0deaf" (UID: "4f826736-e414-4361-8d54-e92088c0deaf"). InnerVolumeSpecName "kube-api-access-cwh6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:18:57.843226 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.843221    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f826736-e414-4361-8d54-e92088c0deaf" (UID: "4f826736-e414-4361-8d54-e92088c0deaf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:18:57.923435 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.923353    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:57.942889 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.942862    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqzfx\" (UniqueName: \"kubernetes.io/projected/3194e8e0-f08e-4475-aaeb-a416d5c64592-kube-api-access-dqzfx\") pod \"3194e8e0-f08e-4475-aaeb-a416d5c64592\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") "
Aug 04 12:18:57.943041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.942909    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3194e8e0-f08e-4475-aaeb-a416d5c64592-plugins-config-volume\") pod \"3194e8e0-f08e-4475-aaeb-a416d5c64592\" (UID: \"3194e8e0-f08e-4475-aaeb-a416d5c64592\") "
Aug 04 12:18:57.943260 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.943236    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3194e8e0-f08e-4475-aaeb-a416d5c64592-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3194e8e0-f08e-4475-aaeb-a416d5c64592" (UID: "3194e8e0-f08e-4475-aaeb-a416d5c64592"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:18:57.943346 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.943246    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f826736-e414-4361-8d54-e92088c0deaf-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:18:57.943346 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.943306    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwh6z\" (UniqueName: \"kubernetes.io/projected/4f826736-e414-4361-8d54-e92088c0deaf-kube-api-access-cwh6z\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:18:57.944855 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:57.944839    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3194e8e0-f08e-4475-aaeb-a416d5c64592-kube-api-access-dqzfx" (OuterVolumeSpecName: "kube-api-access-dqzfx") pod "3194e8e0-f08e-4475-aaeb-a416d5c64592" (UID: "3194e8e0-f08e-4475-aaeb-a416d5c64592"). InnerVolumeSpecName "kube-api-access-dqzfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:18:58.010864 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.010828    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"]
Aug 04 12:18:58.020208 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.020179    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-62afff-65fdcf649b-9t85l"]
Aug 04 12:18:58.044030 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.043999    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqzfx\" (UniqueName: \"kubernetes.io/projected/3194e8e0-f08e-4475-aaeb-a416d5c64592-kube-api-access-dqzfx\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:18:58.044030 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.044027    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3194e8e0-f08e-4475-aaeb-a416d5c64592-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:18:58.271257 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.271222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:18:58.688492 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.688411    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"
Aug 04 12:18:58.688928 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.688410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6" event={"ID":"3194e8e0-f08e-4475-aaeb-a416d5c64592","Type":"ContainerDied","Data":"06cd1693f7a6a68df83f1ff1843904319f5b34e90cf93b8f4455809bb1acded7"}
Aug 04 12:18:58.688928 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.688552    2558 scope.go:117] "RemoveContainer" containerID="01ba689cc9c28f93118fa3eeaf907cc08aa0418545e9d96b61230cf7ef6626e5"
Aug 04 12:18:58.716896 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.716872    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"]
Aug 04 12:18:58.721011 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.720985    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6"]
Aug 04 12:18:58.994639 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.994557    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3194e8e0-f08e-4475-aaeb-a416d5c64592" path="/var/lib/kubelet/pods/3194e8e0-f08e-4475-aaeb-a416d5c64592/volumes"
Aug 04 12:18:58.995005 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:18:58.994991    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f826736-e414-4361-8d54-e92088c0deaf" path="/var/lib/kubelet/pods/4f826736-e414-4361-8d54-e92088c0deaf/volumes"
Aug 04 12:19:04.436082 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:04.436052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:04.520755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:04.520713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:04.804655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:04.804615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:05.027103 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:05.027070    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:07.341628 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:07.341593    2558 scope.go:117] "RemoveContainer" containerID="d9ffa18ea6454cbc767d7048e26fe9c3012d14440d0c43c1cf7a0623068f3c13"
Aug 04 12:19:20.713484 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.713455    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"]
Aug 04 12:19:20.713994 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.713650    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" podUID="5933ad39-2f47-4ff1-81c1-957fc7bef788" containerName="maas-api" containerID="cri-o://9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6" gracePeriod=30
Aug 04 12:19:20.738347 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.738322    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:20.797597 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.797568    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"]
Aug 04 12:19:20.797876 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.797846    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x" podUID="e86ac599-1735-4098-8886-c9faf01bae50" containerName="payload-processing" containerID="cri-o://94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323" gracePeriod=30
Aug 04 12:19:20.807904 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.807875    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"]
Aug 04 12:19:20.808088 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.808067    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn" podUID="8e1b9860-6cef-41d2-a2a1-553d80d35291" containerName="payload-pre-processing" containerID="cri-o://95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7" gracePeriod=30
Aug 04 12:19:20.866086 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.866061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:20.972689 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:20.972659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:21.051681 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.051656    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:19:21.088829 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.088805    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:19:21.095052 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.095022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:21.097096 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.097078    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:19:21.115565 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115541    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls\") pod \"5933ad39-2f47-4ff1-81c1-957fc7bef788\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") "
Aug 04 12:19:21.115712 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115590    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ksln\" (UniqueName: \"kubernetes.io/projected/5933ad39-2f47-4ff1-81c1-957fc7bef788-kube-api-access-2ksln\") pod \"5933ad39-2f47-4ff1-81c1-957fc7bef788\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") "
Aug 04 12:19:21.115712 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115617    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhsl5\" (UniqueName: \"kubernetes.io/projected/8e1b9860-6cef-41d2-a2a1-553d80d35291-kube-api-access-vhsl5\") pod \"8e1b9860-6cef-41d2-a2a1-553d80d35291\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") "
Aug 04 12:19:21.115845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115711    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1b9860-6cef-41d2-a2a1-553d80d35291-plugins-config-volume\") pod \"8e1b9860-6cef-41d2-a2a1-553d80d35291\" (UID: \"8e1b9860-6cef-41d2-a2a1-553d80d35291\") "
Aug 04 12:19:21.115845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115758    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xcmj\" (UniqueName: \"kubernetes.io/projected/e86ac599-1735-4098-8886-c9faf01bae50-kube-api-access-9xcmj\") pod \"e86ac599-1735-4098-8886-c9faf01bae50\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") "
Aug 04 12:19:21.115845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115812    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86ac599-1735-4098-8886-c9faf01bae50-plugins-config-volume\") pod \"e86ac599-1735-4098-8886-c9faf01bae50\" (UID: \"e86ac599-1735-4098-8886-c9faf01bae50\") "
Aug 04 12:19:21.116007 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.115867    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5933ad39-2f47-4ff1-81c1-957fc7bef788-openshift-service-ca\") pod \"5933ad39-2f47-4ff1-81c1-957fc7bef788\" (UID: \"5933ad39-2f47-4ff1-81c1-957fc7bef788\") "
Aug 04 12:19:21.116690 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.116117    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e1b9860-6cef-41d2-a2a1-553d80d35291-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e1b9860-6cef-41d2-a2a1-553d80d35291" (UID: "8e1b9860-6cef-41d2-a2a1-553d80d35291"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:19:21.116690 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.116457    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5933ad39-2f47-4ff1-81c1-957fc7bef788-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5933ad39-2f47-4ff1-81c1-957fc7bef788" (UID: "5933ad39-2f47-4ff1-81c1-957fc7bef788"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:19:21.116690 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.116634    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e86ac599-1735-4098-8886-c9faf01bae50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e86ac599-1735-4098-8886-c9faf01bae50" (UID: "e86ac599-1735-4098-8886-c9faf01bae50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:19:21.118325 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.118260    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1b9860-6cef-41d2-a2a1-553d80d35291-kube-api-access-vhsl5" (OuterVolumeSpecName: "kube-api-access-vhsl5") pod "8e1b9860-6cef-41d2-a2a1-553d80d35291" (UID: "8e1b9860-6cef-41d2-a2a1-553d80d35291"). InnerVolumeSpecName "kube-api-access-vhsl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:19:21.118992 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.118956    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86ac599-1735-4098-8886-c9faf01bae50-kube-api-access-9xcmj" (OuterVolumeSpecName: "kube-api-access-9xcmj") pod "e86ac599-1735-4098-8886-c9faf01bae50" (UID: "e86ac599-1735-4098-8886-c9faf01bae50"). InnerVolumeSpecName "kube-api-access-9xcmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:19:21.119424 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.119389    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5933ad39-2f47-4ff1-81c1-957fc7bef788-kube-api-access-2ksln" (OuterVolumeSpecName: "kube-api-access-2ksln") pod "5933ad39-2f47-4ff1-81c1-957fc7bef788" (UID: "5933ad39-2f47-4ff1-81c1-957fc7bef788"). InnerVolumeSpecName "kube-api-access-2ksln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:19:21.119524 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.119497    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5933ad39-2f47-4ff1-81c1-957fc7bef788" (UID: "5933ad39-2f47-4ff1-81c1-957fc7bef788"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216686    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhsl5\" (UniqueName: \"kubernetes.io/projected/8e1b9860-6cef-41d2-a2a1-553d80d35291-kube-api-access-vhsl5\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216728    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1b9860-6cef-41d2-a2a1-553d80d35291-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216737    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xcmj\" (UniqueName: \"kubernetes.io/projected/e86ac599-1735-4098-8886-c9faf01bae50-kube-api-access-9xcmj\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216746    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86ac599-1735-4098-8886-c9faf01bae50-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216755    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5933ad39-2f47-4ff1-81c1-957fc7bef788-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216764    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5933ad39-2f47-4ff1-81c1-957fc7bef788-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.216787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.216774    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ksln\" (UniqueName: \"kubernetes.io/projected/5933ad39-2f47-4ff1-81c1-957fc7bef788-kube-api-access-2ksln\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:19:21.504695 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.504653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:21.818447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.818360    2558 generic.go:356] "Generic (PLEG): container finished" podID="e86ac599-1735-4098-8886-c9faf01bae50" containerID="94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323" exitCode=0
Aug 04 12:19:21.818447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.818407    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"
Aug 04 12:19:21.818949 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.818443    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x" event={"ID":"e86ac599-1735-4098-8886-c9faf01bae50","Type":"ContainerDied","Data":"94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323"}
Aug 04 12:19:21.818949 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.818501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x" event={"ID":"e86ac599-1735-4098-8886-c9faf01bae50","Type":"ContainerDied","Data":"15068a3e8711ef9aa67dd4c8c4c461794c08f44bf6756c4bebb44dbaade73175"}
Aug 04 12:19:21.818949 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.818536    2558 scope.go:117] "RemoveContainer" containerID="94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323"
Aug 04 12:19:21.819844 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.819770    2558 generic.go:356] "Generic (PLEG): container finished" podID="8e1b9860-6cef-41d2-a2a1-553d80d35291" containerID="95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7" exitCode=0
Aug 04 12:19:21.819844 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.819820    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"
Aug 04 12:19:21.819944 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.819876    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn" event={"ID":"8e1b9860-6cef-41d2-a2a1-553d80d35291","Type":"ContainerDied","Data":"95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7"}
Aug 04 12:19:21.819944 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.819929    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn" event={"ID":"8e1b9860-6cef-41d2-a2a1-553d80d35291","Type":"ContainerDied","Data":"0d828040d060cc16763878ed9117663be7ec526fb01876318ea7295924725803"}
Aug 04 12:19:21.821175 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.821154    2558 generic.go:356] "Generic (PLEG): container finished" podID="5933ad39-2f47-4ff1-81c1-957fc7bef788" containerID="9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6" exitCode=0
Aug 04 12:19:21.821306 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.821193    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" event={"ID":"5933ad39-2f47-4ff1-81c1-957fc7bef788","Type":"ContainerDied","Data":"9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6"}
Aug 04 12:19:21.821306 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.821226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r" event={"ID":"5933ad39-2f47-4ff1-81c1-957fc7bef788","Type":"ContainerDied","Data":"1256a0e20885adf44149cc156eff34e96b6468ab1e84f8782ae47145b63fb8a7"}
Aug 04 12:19:21.821306 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.821197    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"
Aug 04 12:19:21.828824 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.828809    2558 scope.go:117] "RemoveContainer" containerID="94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323"
Aug 04 12:19:21.829096 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:19:21.829074    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323\": container with ID starting with 94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323 not found: ID does not exist" containerID="94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323"
Aug 04 12:19:21.829182 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.829104    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323"} err="failed to get container status \"94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323\": rpc error: code = NotFound desc = could not find container \"94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323\": container with ID starting with 94bc06a4082ec1e0ac71622c8add0449172392ce936ff7826350ca6bfd40e323 not found: ID does not exist"
Aug 04 12:19:21.829182 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.829126    2558 scope.go:117] "RemoveContainer" containerID="95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7"
Aug 04 12:19:21.837381 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.837366    2558 scope.go:117] "RemoveContainer" containerID="95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7"
Aug 04 12:19:21.837592 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:19:21.837575    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7\": container with ID starting with 95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7 not found: ID does not exist" containerID="95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7"
Aug 04 12:19:21.837638 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.837597    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7"} err="failed to get container status \"95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7\": rpc error: code = NotFound desc = could not find container \"95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7\": container with ID starting with 95634c69d1796cf74a89a050fc471247818da6f80b17e8c0e36688baa305ced7 not found: ID does not exist"
Aug 04 12:19:21.837638 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.837612    2558 scope.go:117] "RemoveContainer" containerID="9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6"
Aug 04 12:19:21.844983 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.844965    2558 scope.go:117] "RemoveContainer" containerID="9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6"
Aug 04 12:19:21.845210 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:19:21.845194    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6\": container with ID starting with 9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6 not found: ID does not exist" containerID="9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6"
Aug 04 12:19:21.845267 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.845216    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6"} err="failed to get container status \"9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6\": rpc error: code = NotFound desc = could not find container \"9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6\": container with ID starting with 9e6b48a8a389382fab0133e0f510da85cb08c045d357950674ca51ce04b8dcf6 not found: ID does not exist"
Aug 04 12:19:21.848736 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.848714    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"]
Aug 04 12:19:21.857840 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.857819    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x"]
Aug 04 12:19:21.871603 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.871584    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"]
Aug 04 12:19:21.876496 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.876476    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn"]
Aug 04 12:19:21.891768 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.891749    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"]
Aug 04 12:19:21.895684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:21.895665    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9e17b5-6b74846cb7-qlt6r"]
Aug 04 12:19:22.993970 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:22.993938    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5933ad39-2f47-4ff1-81c1-957fc7bef788" path="/var/lib/kubelet/pods/5933ad39-2f47-4ff1-81c1-957fc7bef788/volumes"
Aug 04 12:19:22.994362 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:22.994274    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e1b9860-6cef-41d2-a2a1-553d80d35291" path="/var/lib/kubelet/pods/8e1b9860-6cef-41d2-a2a1-553d80d35291/volumes"
Aug 04 12:19:22.994608 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:22.994596    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e86ac599-1735-4098-8886-c9faf01bae50" path="/var/lib/kubelet/pods/e86ac599-1735-4098-8886-c9faf01bae50/volumes"
Aug 04 12:19:28.120882 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:28.120849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:28.229727 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:28.229651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:28.423619 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:28.423537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:28.654430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:28.654401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:28.989850 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:28.989820    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:29.341234 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:29.341208    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:29.582844 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:29.582815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:29.809177 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:29.809141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:32.858763 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:32.858732    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:32.959612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:32.959581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:33.949219 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949186    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"]
Aug 04 12:19:33.949632 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949616    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86ac599-1735-4098-8886-c9faf01bae50" containerName="payload-processing"
Aug 04 12:19:33.949678 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949634    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86ac599-1735-4098-8886-c9faf01bae50" containerName="payload-processing"
Aug 04 12:19:33.949678 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949654    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f826736-e414-4361-8d54-e92088c0deaf" containerName="maas-api"
Aug 04 12:19:33.949678 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949660    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f826736-e414-4361-8d54-e92088c0deaf" containerName="maas-api"
Aug 04 12:19:33.949678 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949674    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5933ad39-2f47-4ff1-81c1-957fc7bef788" containerName="maas-api"
Aug 04 12:19:33.949678 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949679    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5933ad39-2f47-4ff1-81c1-957fc7bef788" containerName="maas-api"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949687    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e1b9860-6cef-41d2-a2a1-553d80d35291" containerName="payload-pre-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949692    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e1b9860-6cef-41d2-a2a1-553d80d35291" containerName="payload-pre-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949700    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3194e8e0-f08e-4475-aaeb-a416d5c64592" containerName="payload-pre-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949705    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3194e8e0-f08e-4475-aaeb-a416d5c64592" containerName="payload-pre-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949758    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e1b9860-6cef-41d2-a2a1-553d80d35291" containerName="payload-pre-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949767    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3194e8e0-f08e-4475-aaeb-a416d5c64592" containerName="payload-pre-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949774    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86ac599-1735-4098-8886-c9faf01bae50" containerName="payload-processing"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949781    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f826736-e414-4361-8d54-e92088c0deaf" containerName="maas-api"
Aug 04 12:19:33.949833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.949786    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5933ad39-2f47-4ff1-81c1-957fc7bef788" containerName="maas-api"
Aug 04 12:19:33.954684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.954666    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:33.957383 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.957359    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a7013e-serving-cert\""
Aug 04 12:19:33.964661 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.964639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"]
Aug 04 12:19:33.983809 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:33.983774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:34.028645 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.028617    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrl2n\" (UniqueName: \"kubernetes.io/projected/69a7a597-d783-456f-a35d-14cedc4c4ab9-kube-api-access-nrl2n\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.028802 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.028651    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7a597-d783-456f-a35d-14cedc4c4ab9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.028802 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.028737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7a597-d783-456f-a35d-14cedc4c4ab9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.073604 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.073566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:34.129969 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.129926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7a597-d783-456f-a35d-14cedc4c4ab9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.130107 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.130032    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrl2n\" (UniqueName: \"kubernetes.io/projected/69a7a597-d783-456f-a35d-14cedc4c4ab9-kube-api-access-nrl2n\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.130107 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.130071    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7a597-d783-456f-a35d-14cedc4c4ab9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.130767 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.130727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7a597-d783-456f-a35d-14cedc4c4ab9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.132446 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.132420    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7a597-d783-456f-a35d-14cedc4c4ab9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.140007 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.139986    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrl2n\" (UniqueName: \"kubernetes.io/projected/69a7a597-d783-456f-a35d-14cedc4c4ab9-kube-api-access-nrl2n\") pod \"maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.266043 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.266012    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:34.418355 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.418330    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"]
Aug 04 12:19:34.419381 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:19:34.419354    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a7a597_d783_456f_a35d_14cedc4c4ab9.slice/crio-437c41908473f54d6169cf6814124e5f35e4ba3c13ccb60da1433e078c0422bd WatchSource:0}: Error finding container 437c41908473f54d6169cf6814124e5f35e4ba3c13ccb60da1433e078c0422bd: Status 404 returned error can't find the container with id 437c41908473f54d6169cf6814124e5f35e4ba3c13ccb60da1433e078c0422bd
Aug 04 12:19:34.598699 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.598616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:34.712591 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.712561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:34.889300 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.889256    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr" event={"ID":"69a7a597-d783-456f-a35d-14cedc4c4ab9","Type":"ContainerStarted","Data":"437c41908473f54d6169cf6814124e5f35e4ba3c13ccb60da1433e078c0422bd"}
Aug 04 12:19:34.914117 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.914090    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"]
Aug 04 12:19:34.922095 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.919606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:34.923781 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.923755    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-a7013e-dockercfg-rn9vh\""
Aug 04 12:19:34.923983 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.923969    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-a7013e\""
Aug 04 12:19:34.931795 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:34.931770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"]
Aug 04 12:19:35.038400 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.038360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.038773 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.038511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz82z\" (UniqueName: \"kubernetes.io/projected/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-kube-api-access-wz82z\") pod \"payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.139321 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.139206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.139502 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.139344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz82z\" (UniqueName: \"kubernetes.io/projected/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-kube-api-access-wz82z\") pod \"payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.140141 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.140112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.148400 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.148378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz82z\" (UniqueName: \"kubernetes.io/projected/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-kube-api-access-wz82z\") pod \"payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.233871 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.233839    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.374624 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.374599    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"]
Aug 04 12:19:35.376743 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:19:35.376714    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6251a9a3_faec_4cd0_845e_e9ee3eee82b8.slice/crio-204727c12afbcd0828bcd465b053ef6bb4af417e06c499796800a3e88d5f8bb5 WatchSource:0}: Error finding container 204727c12afbcd0828bcd465b053ef6bb4af417e06c499796800a3e88d5f8bb5: Status 404 returned error can't find the container with id 204727c12afbcd0828bcd465b053ef6bb4af417e06c499796800a3e88d5f8bb5
Aug 04 12:19:35.895772 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.895735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv" event={"ID":"6251a9a3-faec-4cd0-845e-e9ee3eee82b8","Type":"ContainerStarted","Data":"0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12"}
Aug 04 12:19:35.895969 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.895804    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:35.895969 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.895826    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv" event={"ID":"6251a9a3-faec-4cd0-845e-e9ee3eee82b8","Type":"ContainerStarted","Data":"204727c12afbcd0828bcd465b053ef6bb4af417e06c499796800a3e88d5f8bb5"}
Aug 04 12:19:35.897247 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.897221    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr" event={"ID":"69a7a597-d783-456f-a35d-14cedc4c4ab9","Type":"ContainerStarted","Data":"dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013"}
Aug 04 12:19:35.897377 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.897316    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:35.914350 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.914306    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv" podStartSLOduration=1.914268868 podStartE2EDuration="1.914268868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:19:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:19:35.913538788 +0000 UTC m=+2789.484451592" watchObservedRunningTime="2026-08-04 12:19:35.914268868 +0000 UTC m=+2789.485181671"
Aug 04 12:19:35.932372 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:35.932322    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr" podStartSLOduration=2.485637179 podStartE2EDuration="2.932306977s" totalImagesPullingTime="446.669798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:19:35.931405098 +0000 UTC m=+2789.502317902" watchObservedRunningTime="2026-08-04 12:19:35.932306977 +0000 UTC m=+2789.503219774"
Aug 04 12:19:41.907035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:41.907007    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:19:46.904768 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:46.904739    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:19:57.889583 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:57.889555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:58.241679 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:58.241595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:58.558493 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:58.558463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:19:58.680894 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:19:58.680867    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:02.041420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:02.041387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:02.336560 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:02.336453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:03.103868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.103837    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"]
Aug 04 12:20:03.115469 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.115433    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.118777 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.118745    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-038ca6-serving-cert\""
Aug 04 12:20:03.120030 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.120000    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"]
Aug 04 12:20:03.184716 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.184683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ks6z\" (UniqueName: \"kubernetes.io/projected/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-kube-api-access-6ks6z\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.184716 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.184720    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.184993 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.184868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.225051 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.225020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:03.285393 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.285354    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ks6z\" (UniqueName: \"kubernetes.io/projected/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-kube-api-access-6ks6z\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.285393 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.285399    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.285668 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.285489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.285668 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:20:03.285596    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-serving-cert: secret "maas-api-e2e-shared-b-038ca6-serving-cert" not found
Aug 04 12:20:03.285785 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:20:03.285688    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls podName:89cc5de2-fe40-4471-b2b4-a5fb2ff7775b nodeName:}" failed. No retries permitted until 2026-08-04 12:20:03.785664457 +0000 UTC m=+2817.356577239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls") pod "maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" (UID: "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b") : secret "maas-api-e2e-shared-b-038ca6-serving-cert" not found
Aug 04 12:20:03.286067 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.286049    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.299684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.299652    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ks6z\" (UniqueName: \"kubernetes.io/projected/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-kube-api-access-6ks6z\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.760477 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.760442    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:03.790490 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.790447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.793057 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.793032    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls\") pod \"maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:03.879261 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.879227    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"]
Aug 04 12:20:03.882934 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.882916    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:03.885921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.885885    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-038ca6-dockercfg-pb5fb\""
Aug 04 12:20:03.886055 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.885931    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-038ca6\""
Aug 04 12:20:03.891108 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.891079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dlwb\" (UniqueName: \"kubernetes.io/projected/8d624b93-da64-41f5-a52b-44a153e7ca4f-kube-api-access-8dlwb\") pod \"payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:03.891269 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.891250    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d624b93-da64-41f5-a52b-44a153e7ca4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:03.895143 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.895115    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"]
Aug 04 12:20:03.895772 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.895739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:03.992545 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.992509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d624b93-da64-41f5-a52b-44a153e7ca4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:03.992720 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.992554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dlwb\" (UniqueName: \"kubernetes.io/projected/8d624b93-da64-41f5-a52b-44a153e7ca4f-kube-api-access-8dlwb\") pod \"payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:03.993252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:03.993227    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d624b93-da64-41f5-a52b-44a153e7ca4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:04.001834 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:04.001805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dlwb\" (UniqueName: \"kubernetes.io/projected/8d624b93-da64-41f5-a52b-44a153e7ca4f-kube-api-access-8dlwb\") pod \"payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:04.028454 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:04.028368    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:04.171602 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:04.171568    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"]
Aug 04 12:20:04.173208 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:20:04.173185    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cc5de2_fe40_4471_b2b4_a5fb2ff7775b.slice/crio-6a61936b5018d568ee4ad94787edd8bccd7e3c3f30ba0d27fdc0004b703020dc WatchSource:0}: Error finding container 6a61936b5018d568ee4ad94787edd8bccd7e3c3f30ba0d27fdc0004b703020dc: Status 404 returned error can't find the container with id 6a61936b5018d568ee4ad94787edd8bccd7e3c3f30ba0d27fdc0004b703020dc
Aug 04 12:20:04.194921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:04.194885    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:04.336246 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:04.336222    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"]
Aug 04 12:20:04.338009 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:20:04.337983    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d624b93_da64_41f5_a52b_44a153e7ca4f.slice/crio-a796f0b337bda06107cf492d87a48de5edc76e5a9914915ab16c6a61c72d4ffc WatchSource:0}: Error finding container a796f0b337bda06107cf492d87a48de5edc76e5a9914915ab16c6a61c72d4ffc: Status 404 returned error can't find the container with id a796f0b337bda06107cf492d87a48de5edc76e5a9914915ab16c6a61c72d4ffc
Aug 04 12:20:05.049606 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:05.049573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" event={"ID":"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b","Type":"ContainerStarted","Data":"6a61936b5018d568ee4ad94787edd8bccd7e3c3f30ba0d27fdc0004b703020dc"}
Aug 04 12:20:05.051339 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:05.051313    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft" event={"ID":"8d624b93-da64-41f5-a52b-44a153e7ca4f","Type":"ContainerStarted","Data":"5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0"}
Aug 04 12:20:05.051444 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:05.051367    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft" event={"ID":"8d624b93-da64-41f5-a52b-44a153e7ca4f","Type":"ContainerStarted","Data":"a796f0b337bda06107cf492d87a48de5edc76e5a9914915ab16c6a61c72d4ffc"}
Aug 04 12:20:05.051444 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:05.051398    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:05.070248 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:05.070204    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft" podStartSLOduration=2.070188406 podStartE2EDuration="2.070188406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:20:05.068643494 +0000 UTC m=+2818.639556299" watchObservedRunningTime="2026-08-04 12:20:05.070188406 +0000 UTC m=+2818.641101209"
Aug 04 12:20:06.057863 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:06.057822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" event={"ID":"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b","Type":"ContainerStarted","Data":"84399ec6c2cc6ad30c0c2150e2b73eb04052b4bba24643e119770d0b530e3a07"}
Aug 04 12:20:06.058342 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:06.057955    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:06.081308 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:06.081249    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" podStartSLOduration=2.223371321 podStartE2EDuration="3.081233361s" totalImagesPullingTime="857.86204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:20:06.078019918 +0000 UTC m=+2819.648932722" watchObservedRunningTime="2026-08-04 12:20:06.081233361 +0000 UTC m=+2819.652146207"
Aug 04 12:20:12.071306 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:12.071258    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:20:16.060445 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:16.060414    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:20:27.067250 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:27.067218    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:35.696613 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.696578    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"]
Aug 04 12:20:35.701249 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.701229    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.704163 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.704145    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"default-dockercfg-qlmqv\""
Aug 04 12:20:35.704272 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.704183    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"kube-root-ca.crt\""
Aug 04 12:20:35.705429 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.705401    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"openshift-service-ca.crt\""
Aug 04 12:20:35.705497 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.705429    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"auth-test-model-a7013e-kserve-self-signed-certs\""
Aug 04 12:20:35.713401 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.713370    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"]
Aug 04 12:20:35.727490 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.727466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:20:35.780373 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.780336    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-dshm\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.780607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.780427    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-model-cache\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.780607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.780455    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tmp-dir\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.780607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.780485    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ml9\" (UniqueName: \"kubernetes.io/projected/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-kube-api-access-m5ml9\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.780607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.780536    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-home\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.780607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.780561    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tls-certs\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.881792 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.881752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-model-cache\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.881792 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.881792    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tmp-dir\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882070 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.881827    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ml9\" (UniqueName: \"kubernetes.io/projected/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-kube-api-access-m5ml9\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882070 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.881944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-home\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882070 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.881988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tls-certs\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882222 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.882113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-dshm\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882266 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.882218    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-model-cache\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882348 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.882275    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tmp-dir\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.882348 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.882321    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-home\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.884330 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.884303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-dshm\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.884716 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.884693    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tls-certs\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:35.891068 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:35.891042    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ml9\" (UniqueName: \"kubernetes.io/projected/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-kube-api-access-m5ml9\") pod \"auth-test-model-a7013e-kserve-6997cb877f-zv5w9\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") " pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:36.010450 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:36.010411    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:36.151787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:36.151760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"]
Aug 04 12:20:36.153915 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:20:36.153881    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11b8f6df_f6ea_41e2_8a2a_2b5abb67e42d.slice/crio-d6235a416390f37ecac6a3fc844480fc83ae2182cc2f37a4d47dff86581fe5f6 WatchSource:0}: Error finding container d6235a416390f37ecac6a3fc844480fc83ae2182cc2f37a4d47dff86581fe5f6: Status 404 returned error can't find the container with id d6235a416390f37ecac6a3fc844480fc83ae2182cc2f37a4d47dff86581fe5f6
Aug 04 12:20:36.214336 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:36.214310    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9" event={"ID":"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d","Type":"ContainerStarted","Data":"d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a"}
Aug 04 12:20:36.214471 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:36.214351    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9" event={"ID":"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d","Type":"ContainerStarted","Data":"d6235a416390f37ecac6a3fc844480fc83ae2182cc2f37a4d47dff86581fe5f6"}
Aug 04 12:20:36.233035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:36.232985    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9" podStartSLOduration=1.232970158 podStartE2EDuration="1.232970158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:20:36.2307794 +0000 UTC m=+2849.801692207" watchObservedRunningTime="2026-08-04 12:20:36.232970158 +0000 UTC m=+2849.803882961"
Aug 04 12:20:46.010891 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:46.010852    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:46.011374 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:46.010907    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:46.023557 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:46.023531    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:20:46.274847 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:20:46.274759    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:21:06.711236 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:21:06.711154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:21:26.027535 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:21:26.027504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:21:29.191218 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:21:29.191179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:22:07.398140 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:22:07.398108    2558 scope.go:117] "RemoveContainer" containerID="00357d2d23d9fdc9e1d63d864eb67b53090240d1e703e889b21645a980c90e97"
Aug 04 12:22:52.958836 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:22:52.958713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:22:53.025822 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:22:53.025774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:23:07.160596 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:23:07.160564    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:23:07.170453 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:23:07.170435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:24:24.389026 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:24:24.388918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:25:19.242567 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:19.242531    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"]
Aug 04 12:25:19.261629 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:19.261599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:25:20.204726 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:20.204696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:25:34.261491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.261437    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9" podUID="11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" containerName="main" containerID="cri-o://d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a" gracePeriod=45
Aug 04 12:25:34.510309 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.510273    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:25:34.604864 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.604791    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-dshm\") pod \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") "
Aug 04 12:25:34.604864 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.604856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tls-certs\") pod \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") "
Aug 04 12:25:34.605093 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.604877    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5ml9\" (UniqueName: \"kubernetes.io/projected/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-kube-api-access-m5ml9\") pod \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") "
Aug 04 12:25:34.605093 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.604924    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-home\") pod \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") "
Aug 04 12:25:34.605093 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605014    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-model-cache\") pod \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") "
Aug 04 12:25:34.605093 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605057    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tmp-dir\") pod \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\" (UID: \"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d\") "
Aug 04 12:25:34.605344 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605168    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-home" (OuterVolumeSpecName: "home") pod "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" (UID: "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:25:34.605344 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605269    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-model-cache" (OuterVolumeSpecName: "model-cache") pod "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" (UID: "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:25:34.605455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605391    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" (UID: "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:25:34.605455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605406    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:25:34.605455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.605442    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:25:34.607197 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.607169    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-kube-api-access-m5ml9" (OuterVolumeSpecName: "kube-api-access-m5ml9") pod "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" (UID: "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d"). InnerVolumeSpecName "kube-api-access-m5ml9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:25:34.607197 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.607180    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" (UID: "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:25:34.607377 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.607368    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-dshm" (OuterVolumeSpecName: "dshm") pod "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" (UID: "11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:25:34.706756 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.706724    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:25:34.706756 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.706750    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:25:34.706756 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.706761    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5ml9\" (UniqueName: \"kubernetes.io/projected/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-kube-api-access-m5ml9\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:25:34.707020 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.706792    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:25:34.750821 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.750792    2558 generic.go:356] "Generic (PLEG): container finished" podID="11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" containerID="d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a" exitCode=0
Aug 04 12:25:34.750971 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.750839    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"
Aug 04 12:25:34.750971 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.750872    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9" event={"ID":"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d","Type":"ContainerDied","Data":"d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a"}
Aug 04 12:25:34.750971 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.750918    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9" event={"ID":"11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d","Type":"ContainerDied","Data":"d6235a416390f37ecac6a3fc844480fc83ae2182cc2f37a4d47dff86581fe5f6"}
Aug 04 12:25:34.750971 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.750936    2558 scope.go:117] "RemoveContainer" containerID="d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a"
Aug 04 12:25:34.760812 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.760798    2558 scope.go:117] "RemoveContainer" containerID="d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a"
Aug 04 12:25:34.761030 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:25:34.761015    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a\": container with ID starting with d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a not found: ID does not exist" containerID="d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a"
Aug 04 12:25:34.761081 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.761037    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a"} err="failed to get container status \"d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a\": rpc error: code = NotFound desc = could not find container \"d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a\": container with ID starting with d535c90f679f46a2a899fc7050d151587fe0c8327004387a891285329a9a8d1a not found: ID does not exist"
Aug 04 12:25:34.781261 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.781237    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"]
Aug 04 12:25:34.786158 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.786137    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/auth-test-model-a7013e-kserve-6997cb877f-zv5w9"]
Aug 04 12:25:34.994476 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:34.994381    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" path="/var/lib/kubelet/pods/11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d/volumes"
Aug 04 12:25:44.022849 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.022815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:25:44.569060 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.569023    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"]
Aug 04 12:25:44.569548 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.569527    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" containerName="main"
Aug 04 12:25:44.569548 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.569548    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" containerName="main"
Aug 04 12:25:44.569717 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.569636    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b8f6df-f6ea-41e2-8a2a-2b5abb67e42d" containerName="main"
Aug 04 12:25:44.573893 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.573875    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.581627 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.581605    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"kube-root-ca.crt\""
Aug 04 12:25:44.581727 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.581625    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"sub-test-model-a7013e-kserve-self-signed-certs\""
Aug 04 12:25:44.581984 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.581969    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"openshift-service-ca.crt\""
Aug 04 12:25:44.582031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.581975    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"default-dockercfg-qlmqv\""
Aug 04 12:25:44.592308 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.592269    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"]
Aug 04 12:25:44.600431 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.600396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:25:44.696615 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.696586    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-dshm\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.696776 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.696628    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-home\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.696776 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.696652    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-tmp-dir\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.696776 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.696689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f96vh\" (UniqueName: \"kubernetes.io/projected/32def9fc-257b-467b-849a-dc105f8165d0-kube-api-access-f96vh\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.696888 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.696797    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-model-cache\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.696888 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.696832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32def9fc-257b-467b-849a-dc105f8165d0-tls-certs\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.797865 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.797835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-home\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.797872    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-tmp-dir\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.797893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f96vh\" (UniqueName: \"kubernetes.io/projected/32def9fc-257b-467b-849a-dc105f8165d0-kube-api-access-f96vh\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.797937    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-model-cache\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.797963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32def9fc-257b-467b-849a-dc105f8165d0-tls-certs\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.798024    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-dshm\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798311 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.798218    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-home\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798311 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.798265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-tmp-dir\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.798437 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.798358    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-model-cache\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.800427 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.800407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-dshm\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.800727 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.800709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32def9fc-257b-467b-849a-dc105f8165d0-tls-certs\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.807933 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.807914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f96vh\" (UniqueName: \"kubernetes.io/projected/32def9fc-257b-467b-849a-dc105f8165d0-kube-api-access-f96vh\") pod \"sub-test-model-a7013e-kserve-7b67d98cb9-k4p94\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") " pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:44.883053 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:44.882981    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:45.021129 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:45.021104    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"]
Aug 04 12:25:45.022396 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:25:45.022369    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32def9fc_257b_467b_849a_dc105f8165d0.slice/crio-9f3322af2c5141029fb1591f2b7cd34f1da20ce4ee6952aca88afbe2b060cf7d WatchSource:0}: Error finding container 9f3322af2c5141029fb1591f2b7cd34f1da20ce4ee6952aca88afbe2b060cf7d: Status 404 returned error can't find the container with id 9f3322af2c5141029fb1591f2b7cd34f1da20ce4ee6952aca88afbe2b060cf7d
Aug 04 12:25:45.810139 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:45.810103    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94" event={"ID":"32def9fc-257b-467b-849a-dc105f8165d0","Type":"ContainerStarted","Data":"b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14"}
Aug 04 12:25:45.810532 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:45.810154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94" event={"ID":"32def9fc-257b-467b-849a-dc105f8165d0","Type":"ContainerStarted","Data":"9f3322af2c5141029fb1591f2b7cd34f1da20ce4ee6952aca88afbe2b060cf7d"}
Aug 04 12:25:45.839190 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:45.839139    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94" podStartSLOduration=1.839124127 podStartE2EDuration="1.839124127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:25:45.837412101 +0000 UTC m=+3159.408324919" watchObservedRunningTime="2026-08-04 12:25:45.839124127 +0000 UTC m=+3159.410036931"
Aug 04 12:25:54.883745 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:54.883714    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:54.884256 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:54.883999    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:54.896200 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:54.896175    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:25:55.871417 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:25:55.871386    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:26:00.140543 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:26:00.140510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:27:08.239591 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:08.239501    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"]
Aug 04 12:27:08.266888 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:08.266850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:27:09.516039 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:09.516000    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:27:23.258205 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.258164    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94" podUID="32def9fc-257b-467b-849a-dc105f8165d0" containerName="main" containerID="cri-o://b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14" gracePeriod=45
Aug 04 12:27:23.501417 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.501390    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:27:23.586398 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586326    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-tmp-dir\") pod \"32def9fc-257b-467b-849a-dc105f8165d0\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") "
Aug 04 12:27:23.586398 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586361    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f96vh\" (UniqueName: \"kubernetes.io/projected/32def9fc-257b-467b-849a-dc105f8165d0-kube-api-access-f96vh\") pod \"32def9fc-257b-467b-849a-dc105f8165d0\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") "
Aug 04 12:27:23.586618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586430    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-model-cache\") pod \"32def9fc-257b-467b-849a-dc105f8165d0\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") "
Aug 04 12:27:23.586618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586456    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-dshm\") pod \"32def9fc-257b-467b-849a-dc105f8165d0\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") "
Aug 04 12:27:23.586618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586497    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-home\") pod \"32def9fc-257b-467b-849a-dc105f8165d0\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") "
Aug 04 12:27:23.586618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586559    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32def9fc-257b-467b-849a-dc105f8165d0-tls-certs\") pod \"32def9fc-257b-467b-849a-dc105f8165d0\" (UID: \"32def9fc-257b-467b-849a-dc105f8165d0\") "
Aug 04 12:27:23.586618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586584    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32def9fc-257b-467b-849a-dc105f8165d0" (UID: "32def9fc-257b-467b-849a-dc105f8165d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:27:23.586912 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586683    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "32def9fc-257b-467b-849a-dc105f8165d0" (UID: "32def9fc-257b-467b-849a-dc105f8165d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:27:23.586912 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586825    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-home" (OuterVolumeSpecName: "home") pod "32def9fc-257b-467b-849a-dc105f8165d0" (UID: "32def9fc-257b-467b-849a-dc105f8165d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:27:23.587038 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586928    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:27:23.587038 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586947    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:27:23.587038 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.586959    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:27:23.588644 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.588616    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-dshm" (OuterVolumeSpecName: "dshm") pod "32def9fc-257b-467b-849a-dc105f8165d0" (UID: "32def9fc-257b-467b-849a-dc105f8165d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:27:23.588755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.588640    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32def9fc-257b-467b-849a-dc105f8165d0-kube-api-access-f96vh" (OuterVolumeSpecName: "kube-api-access-f96vh") pod "32def9fc-257b-467b-849a-dc105f8165d0" (UID: "32def9fc-257b-467b-849a-dc105f8165d0"). InnerVolumeSpecName "kube-api-access-f96vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:27:23.588755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.588684    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32def9fc-257b-467b-849a-dc105f8165d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32def9fc-257b-467b-849a-dc105f8165d0" (UID: "32def9fc-257b-467b-849a-dc105f8165d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:27:23.688059 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.688034    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32def9fc-257b-467b-849a-dc105f8165d0-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:27:23.688059 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.688061    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f96vh\" (UniqueName: \"kubernetes.io/projected/32def9fc-257b-467b-849a-dc105f8165d0-kube-api-access-f96vh\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:27:23.688222 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:23.688074    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32def9fc-257b-467b-849a-dc105f8165d0-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:27:24.304345 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.304308    2558 generic.go:356] "Generic (PLEG): container finished" podID="32def9fc-257b-467b-849a-dc105f8165d0" containerID="b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14" exitCode=0
Aug 04 12:27:24.304345 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.304348    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"
Aug 04 12:27:24.304846 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.304375    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94" event={"ID":"32def9fc-257b-467b-849a-dc105f8165d0","Type":"ContainerDied","Data":"b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14"}
Aug 04 12:27:24.304846 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.304412    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94" event={"ID":"32def9fc-257b-467b-849a-dc105f8165d0","Type":"ContainerDied","Data":"9f3322af2c5141029fb1591f2b7cd34f1da20ce4ee6952aca88afbe2b060cf7d"}
Aug 04 12:27:24.304846 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.304431    2558 scope.go:117] "RemoveContainer" containerID="b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14"
Aug 04 12:27:24.314389 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.314371    2558 scope.go:117] "RemoveContainer" containerID="b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14"
Aug 04 12:27:24.314667 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:27:24.314645    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14\": container with ID starting with b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14 not found: ID does not exist" containerID="b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14"
Aug 04 12:27:24.314734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.314675    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14"} err="failed to get container status \"b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14\": rpc error: code = NotFound desc = could not find container \"b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14\": container with ID starting with b3e6c2874073aac3d500bf2919d5f9ddee2222509cf094bd70eaad363d9eab14 not found: ID does not exist"
Aug 04 12:27:24.328236 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.328211    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"]
Aug 04 12:27:24.334753 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.334728    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/sub-test-model-a7013e-kserve-7b67d98cb9-k4p94"]
Aug 04 12:27:24.994447 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:24.994416    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32def9fc-257b-467b-849a-dc105f8165d0" path="/var/lib/kubelet/pods/32def9fc-257b-467b-849a-dc105f8165d0/volumes"
Aug 04 12:27:30.493052 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.493020    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"]
Aug 04 12:27:30.493446 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.493423    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32def9fc-257b-467b-849a-dc105f8165d0" containerName="main"
Aug 04 12:27:30.493446 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.493435    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="32def9fc-257b-467b-849a-dc105f8165d0" containerName="main"
Aug 04 12:27:30.493531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.493503    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="32def9fc-257b-467b-849a-dc105f8165d0" containerName="main"
Aug 04 12:27:30.497826 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.497809    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.501197 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.501177    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"openshift-service-ca.crt\""
Aug 04 12:27:30.501333 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.501253    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"kube-root-ca.crt\""
Aug 04 12:27:30.501333 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.501257    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"default-dockercfg-qlmqv\""
Aug 04 12:27:30.502367 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.502350    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7013e\"/\"rate-test-model-a7013e-kserve-self-signed-certs\""
Aug 04 12:27:30.507839 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.507821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"]
Aug 04 12:27:30.528677 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.528653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:27:30.648041 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.648009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-dshm\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.648183 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.648046    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-home\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.648183 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.648119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/804e05f8-bf25-49e9-a1c6-736302948b5b-tls-certs\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.648183 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.648152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqgxv\" (UniqueName: \"kubernetes.io/projected/804e05f8-bf25-49e9-a1c6-736302948b5b-kube-api-access-cqgxv\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.648323 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.648188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-model-cache\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.648323 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.648203    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-tmp-dir\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.749523 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-dshm\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.749787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-home\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.749787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/804e05f8-bf25-49e9-a1c6-736302948b5b-tls-certs\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.749787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749628    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqgxv\" (UniqueName: \"kubernetes.io/projected/804e05f8-bf25-49e9-a1c6-736302948b5b-kube-api-access-cqgxv\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.749787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-model-cache\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.749787 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-tmp-dir\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.750106 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.749969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-home\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.750106 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.750069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-tmp-dir\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.750195 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.750165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-model-cache\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.751862 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.751833    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-dshm\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.752357 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.752339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/804e05f8-bf25-49e9-a1c6-736302948b5b-tls-certs\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.758356 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.758338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqgxv\" (UniqueName: \"kubernetes.io/projected/804e05f8-bf25-49e9-a1c6-736302948b5b-kube-api-access-cqgxv\") pod \"rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") " pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.807684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.807651    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:30.944079 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:30.944052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"]
Aug 04 12:27:30.946161 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:27:30.946130    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod804e05f8_bf25_49e9_a1c6_736302948b5b.slice/crio-9a9a518febeaeb862dea15db20ed5fa374b70333794917509e4e52ab6f5a1a80 WatchSource:0}: Error finding container 9a9a518febeaeb862dea15db20ed5fa374b70333794917509e4e52ab6f5a1a80: Status 404 returned error can't find the container with id 9a9a518febeaeb862dea15db20ed5fa374b70333794917509e4e52ab6f5a1a80
Aug 04 12:27:31.344491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:31.344396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7" event={"ID":"804e05f8-bf25-49e9-a1c6-736302948b5b","Type":"ContainerStarted","Data":"f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696"}
Aug 04 12:27:31.344491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:31.344448    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7" event={"ID":"804e05f8-bf25-49e9-a1c6-736302948b5b","Type":"ContainerStarted","Data":"9a9a518febeaeb862dea15db20ed5fa374b70333794917509e4e52ab6f5a1a80"}
Aug 04 12:27:31.367747 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:31.367687    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7" podStartSLOduration=1.367672741 podStartE2EDuration="1.367672741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:27:31.364950749 +0000 UTC m=+3264.935863553" watchObservedRunningTime="2026-08-04 12:27:31.367672741 +0000 UTC m=+3264.938585546"
Aug 04 12:27:40.807857 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:40.807818    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:40.808409 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:40.807872    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:40.820577 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:40.820557    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:41.357946 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:41.357908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:27:41.404407 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:41.404381    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:27:41.422718 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:27:41.422689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:28:01.822084 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.822039    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"]
Aug 04 12:28:01.825735 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.825710    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:01.830464 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.830443    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-038ca6\"/\"rate-test-model-038ca6-kserve-self-signed-certs\""
Aug 04 12:28:01.830612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.830444    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-038ca6\"/\"openshift-service-ca.crt\""
Aug 04 12:28:01.830612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.830444    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-038ca6\"/\"kube-root-ca.crt\""
Aug 04 12:28:01.830612 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.830444    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-038ca6\"/\"default-dockercfg-gjpvc\""
Aug 04 12:28:01.837032 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.837003    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"]
Aug 04 12:28:01.935704 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.935665    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d614265d-07bb-4073-8f9d-725d7592bfb5-tls-certs\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:01.935704 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.935704    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-home\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:01.935942 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.935765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-tmp-dir\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:01.935942 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.935796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd5pz\" (UniqueName: \"kubernetes.io/projected/d614265d-07bb-4073-8f9d-725d7592bfb5-kube-api-access-nd5pz\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:01.935942 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.935813    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-dshm\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:01.935942 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:01.935861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-model-cache\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.036967 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.036927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-model-cache\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.036967 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.036974    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d614265d-07bb-4073-8f9d-725d7592bfb5-tls-certs\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037250 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.036994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-home\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037250 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.037137    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-tmp-dir\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037250 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.037201    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd5pz\" (UniqueName: \"kubernetes.io/projected/d614265d-07bb-4073-8f9d-725d7592bfb5-kube-api-access-nd5pz\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037250 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.037234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-dshm\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037510 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.037414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-home\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037510 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.037474    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-model-cache\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.037510 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.037493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-tmp-dir\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.039470 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.039448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-dshm\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.039773 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.039751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d614265d-07bb-4073-8f9d-725d7592bfb5-tls-certs\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.047181 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.047159    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd5pz\" (UniqueName: \"kubernetes.io/projected/d614265d-07bb-4073-8f9d-725d7592bfb5-kube-api-access-nd5pz\") pod \"rate-test-model-038ca6-kserve-76f7fd489f-v4wr5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") " pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.137264 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.137175    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:02.280989 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.280962    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"]
Aug 04 12:28:02.282763 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:28:02.282735    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd614265d_07bb_4073_8f9d_725d7592bfb5.slice/crio-a2f994386876ff49d47f8918070ea1cf9a9960ca3f31d73f6f9b28878cdede5d WatchSource:0}: Error finding container a2f994386876ff49d47f8918070ea1cf9a9960ca3f31d73f6f9b28878cdede5d: Status 404 returned error can't find the container with id a2f994386876ff49d47f8918070ea1cf9a9960ca3f31d73f6f9b28878cdede5d
Aug 04 12:28:02.503326 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.503270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5" event={"ID":"d614265d-07bb-4073-8f9d-725d7592bfb5","Type":"ContainerStarted","Data":"668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813"}
Aug 04 12:28:02.503491 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.503336    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5" event={"ID":"d614265d-07bb-4073-8f9d-725d7592bfb5","Type":"ContainerStarted","Data":"a2f994386876ff49d47f8918070ea1cf9a9960ca3f31d73f6f9b28878cdede5d"}
Aug 04 12:28:02.523438 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.523388    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5" podStartSLOduration=1.523371976 podStartE2EDuration="1.523371976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:28:02.521863084 +0000 UTC m=+3296.092775888" watchObservedRunningTime="2026-08-04 12:28:02.523371976 +0000 UTC m=+3296.094284780"
Aug 04 12:28:02.871223 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:02.871122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:28:07.194801 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:07.194764    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:28:07.205340 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:07.205316    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:28:12.137810 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:12.137770    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:12.138356 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:12.137957    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:12.150499 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:12.150479    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:12.566028 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:12.565999    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:28:16.270469 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:16.270438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:28:27.509077 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:28:27.509038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:14.256329 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:14.256267    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"]
Aug 04 12:29:14.270768 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:14.270734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:15.318768 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:15.318737    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"]
Aug 04 12:29:15.336614 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:15.336582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:29.275731 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.275695    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7" podUID="804e05f8-bf25-49e9-a1c6-736302948b5b" containerName="main" containerID="cri-o://f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696" gracePeriod=45
Aug 04 12:29:29.541533 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.541506    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:29:29.624775 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.624745    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/804e05f8-bf25-49e9-a1c6-736302948b5b-tls-certs\") pod \"804e05f8-bf25-49e9-a1c6-736302948b5b\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") "
Aug 04 12:29:29.624966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.624782    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-dshm\") pod \"804e05f8-bf25-49e9-a1c6-736302948b5b\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") "
Aug 04 12:29:29.624966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.624863    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-home\") pod \"804e05f8-bf25-49e9-a1c6-736302948b5b\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") "
Aug 04 12:29:29.624966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.624892    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqgxv\" (UniqueName: \"kubernetes.io/projected/804e05f8-bf25-49e9-a1c6-736302948b5b-kube-api-access-cqgxv\") pod \"804e05f8-bf25-49e9-a1c6-736302948b5b\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") "
Aug 04 12:29:29.624966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.624910    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-tmp-dir\") pod \"804e05f8-bf25-49e9-a1c6-736302948b5b\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") "
Aug 04 12:29:29.624966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.624943    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-model-cache\") pod \"804e05f8-bf25-49e9-a1c6-736302948b5b\" (UID: \"804e05f8-bf25-49e9-a1c6-736302948b5b\") "
Aug 04 12:29:29.625246 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.625153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-home" (OuterVolumeSpecName: "home") pod "804e05f8-bf25-49e9-a1c6-736302948b5b" (UID: "804e05f8-bf25-49e9-a1c6-736302948b5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:29.625246 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.625219    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "804e05f8-bf25-49e9-a1c6-736302948b5b" (UID: "804e05f8-bf25-49e9-a1c6-736302948b5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:29.625362 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.625332    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "804e05f8-bf25-49e9-a1c6-736302948b5b" (UID: "804e05f8-bf25-49e9-a1c6-736302948b5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:29.626893 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.626872    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-dshm" (OuterVolumeSpecName: "dshm") pod "804e05f8-bf25-49e9-a1c6-736302948b5b" (UID: "804e05f8-bf25-49e9-a1c6-736302948b5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:29.627021 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.626999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/804e05f8-bf25-49e9-a1c6-736302948b5b-kube-api-access-cqgxv" (OuterVolumeSpecName: "kube-api-access-cqgxv") pod "804e05f8-bf25-49e9-a1c6-736302948b5b" (UID: "804e05f8-bf25-49e9-a1c6-736302948b5b"). InnerVolumeSpecName "kube-api-access-cqgxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:29:29.627078 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.627046    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/804e05f8-bf25-49e9-a1c6-736302948b5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "804e05f8-bf25-49e9-a1c6-736302948b5b" (UID: "804e05f8-bf25-49e9-a1c6-736302948b5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:29:29.726617 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.726590    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:29.726617 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.726613    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/804e05f8-bf25-49e9-a1c6-736302948b5b-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:29.726617 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.726621    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:29.726818 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.726629    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:29.726818 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.726638    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqgxv\" (UniqueName: \"kubernetes.io/projected/804e05f8-bf25-49e9-a1c6-736302948b5b-kube-api-access-cqgxv\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:29.726818 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.726646    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/804e05f8-bf25-49e9-a1c6-736302948b5b-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:29.963814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.963720    2558 generic.go:356] "Generic (PLEG): container finished" podID="804e05f8-bf25-49e9-a1c6-736302948b5b" containerID="f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696" exitCode=0
Aug 04 12:29:29.963814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.963765    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"
Aug 04 12:29:29.964060 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.963813    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7" event={"ID":"804e05f8-bf25-49e9-a1c6-736302948b5b","Type":"ContainerDied","Data":"f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696"}
Aug 04 12:29:29.964060 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.963863    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7" event={"ID":"804e05f8-bf25-49e9-a1c6-736302948b5b","Type":"ContainerDied","Data":"9a9a518febeaeb862dea15db20ed5fa374b70333794917509e4e52ab6f5a1a80"}
Aug 04 12:29:29.964060 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.963885    2558 scope.go:117] "RemoveContainer" containerID="f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696"
Aug 04 12:29:29.973226 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.973208    2558 scope.go:117] "RemoveContainer" containerID="f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696"
Aug 04 12:29:29.973500 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:29:29.973481    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696\": container with ID starting with f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696 not found: ID does not exist" containerID="f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696"
Aug 04 12:29:29.973581 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.973510    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696"} err="failed to get container status \"f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696\": rpc error: code = NotFound desc = could not find container \"f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696\": container with ID starting with f21953ae3c3a08097a6a3009f5bdc4cd19fa9ad72d46e21e082bc28a70941696 not found: ID does not exist"
Aug 04 12:29:29.986888 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.986865    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"]
Aug 04 12:29:29.990901 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:29.990874    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a7013e/rate-test-model-a7013e-kserve-7f8785cfc9-9bjj7"]
Aug 04 12:29:30.337801 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.337763    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5" podUID="d614265d-07bb-4073-8f9d-725d7592bfb5" containerName="main" containerID="cri-o://668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813" gracePeriod=45
Aug 04 12:29:30.589744 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.589688    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:29:30.635132 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635102    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-tmp-dir\") pod \"d614265d-07bb-4073-8f9d-725d7592bfb5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") "
Aug 04 12:29:30.635303 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635146    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d614265d-07bb-4073-8f9d-725d7592bfb5-tls-certs\") pod \"d614265d-07bb-4073-8f9d-725d7592bfb5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") "
Aug 04 12:29:30.635303 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635206    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-model-cache\") pod \"d614265d-07bb-4073-8f9d-725d7592bfb5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") "
Aug 04 12:29:30.635303 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635226    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd5pz\" (UniqueName: \"kubernetes.io/projected/d614265d-07bb-4073-8f9d-725d7592bfb5-kube-api-access-nd5pz\") pod \"d614265d-07bb-4073-8f9d-725d7592bfb5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") "
Aug 04 12:29:30.635499 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635425    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-home\") pod \"d614265d-07bb-4073-8f9d-725d7592bfb5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") "
Aug 04 12:29:30.635499 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635468    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-dshm\") pod \"d614265d-07bb-4073-8f9d-725d7592bfb5\" (UID: \"d614265d-07bb-4073-8f9d-725d7592bfb5\") "
Aug 04 12:29:30.635499 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635469    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d614265d-07bb-4073-8f9d-725d7592bfb5" (UID: "d614265d-07bb-4073-8f9d-725d7592bfb5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:30.635661 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635583    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "d614265d-07bb-4073-8f9d-725d7592bfb5" (UID: "d614265d-07bb-4073-8f9d-725d7592bfb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:30.635723 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635701    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-home" (OuterVolumeSpecName: "home") pod "d614265d-07bb-4073-8f9d-725d7592bfb5" (UID: "d614265d-07bb-4073-8f9d-725d7592bfb5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:30.635893 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635829    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:30.635893 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635852    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:30.635893 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.635869    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:30.637655 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.637630    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d614265d-07bb-4073-8f9d-725d7592bfb5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d614265d-07bb-4073-8f9d-725d7592bfb5" (UID: "d614265d-07bb-4073-8f9d-725d7592bfb5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:29:30.637874 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.637850    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-dshm" (OuterVolumeSpecName: "dshm") pod "d614265d-07bb-4073-8f9d-725d7592bfb5" (UID: "d614265d-07bb-4073-8f9d-725d7592bfb5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:29:30.637992 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.637972    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d614265d-07bb-4073-8f9d-725d7592bfb5-kube-api-access-nd5pz" (OuterVolumeSpecName: "kube-api-access-nd5pz") pod "d614265d-07bb-4073-8f9d-725d7592bfb5" (UID: "d614265d-07bb-4073-8f9d-725d7592bfb5"). InnerVolumeSpecName "kube-api-access-nd5pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:29:30.736601 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.736574    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d614265d-07bb-4073-8f9d-725d7592bfb5-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:30.736601 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.736597    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d614265d-07bb-4073-8f9d-725d7592bfb5-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:30.736601 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.736609    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd5pz\" (UniqueName: \"kubernetes.io/projected/d614265d-07bb-4073-8f9d-725d7592bfb5-kube-api-access-nd5pz\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:30.970729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.970646    2558 generic.go:356] "Generic (PLEG): container finished" podID="d614265d-07bb-4073-8f9d-725d7592bfb5" containerID="668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813" exitCode=0
Aug 04 12:29:30.970729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.970690    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"
Aug 04 12:29:30.970921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.970727    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5" event={"ID":"d614265d-07bb-4073-8f9d-725d7592bfb5","Type":"ContainerDied","Data":"668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813"}
Aug 04 12:29:30.970921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.970771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5" event={"ID":"d614265d-07bb-4073-8f9d-725d7592bfb5","Type":"ContainerDied","Data":"a2f994386876ff49d47f8918070ea1cf9a9960ca3f31d73f6f9b28878cdede5d"}
Aug 04 12:29:30.970921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.970794    2558 scope.go:117] "RemoveContainer" containerID="668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813"
Aug 04 12:29:30.981551 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.981494    2558 scope.go:117] "RemoveContainer" containerID="668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813"
Aug 04 12:29:30.981868 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:29:30.981844    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813\": container with ID starting with 668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813 not found: ID does not exist" containerID="668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813"
Aug 04 12:29:30.981916 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.981881    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813"} err="failed to get container status \"668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813\": rpc error: code = NotFound desc = could not find container \"668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813\": container with ID starting with 668ecbed36801b506c6384f0937d9563d768f737001bdf494d2ac58f95ae1813 not found: ID does not exist"
Aug 04 12:29:30.994251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.994224    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="804e05f8-bf25-49e9-a1c6-736302948b5b" path="/var/lib/kubelet/pods/804e05f8-bf25-49e9-a1c6-736302948b5b/volumes"
Aug 04 12:29:30.994511 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.994497    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"]
Aug 04 12:29:30.998230 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:30.998209    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-038ca6/rate-test-model-038ca6-kserve-76f7fd489f-v4wr5"]
Aug 04 12:29:32.993925 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:32.993894    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d614265d-07bb-4073-8f9d-725d7592bfb5" path="/var/lib/kubelet/pods/d614265d-07bb-4073-8f9d-725d7592bfb5/volumes"
Aug 04 12:29:33.571316 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571272    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:33.571674 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571662    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d614265d-07bb-4073-8f9d-725d7592bfb5" containerName="main"
Aug 04 12:29:33.571732 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571676    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d614265d-07bb-4073-8f9d-725d7592bfb5" containerName="main"
Aug 04 12:29:33.571732 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571700    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="804e05f8-bf25-49e9-a1c6-736302948b5b" containerName="main"
Aug 04 12:29:33.571732 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571706    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="804e05f8-bf25-49e9-a1c6-736302948b5b" containerName="main"
Aug 04 12:29:33.571821 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571767    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="804e05f8-bf25-49e9-a1c6-736302948b5b" containerName="main"
Aug 04 12:29:33.571821 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.571775    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d614265d-07bb-4073-8f9d-725d7592bfb5" containerName="main"
Aug 04 12:29:33.576088 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.576064    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:33.579865 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.579848    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bcgns\""
Aug 04 12:29:33.586852 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.586829    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:33.661558 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.661521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgrx5\" (UniqueName: \"kubernetes.io/projected/ef04ec97-4718-4f7b-9aae-171d95731378-kube-api-access-lgrx5\") pod \"test-curl-7470-140200320999760\" (UID: \"ef04ec97-4718-4f7b-9aae-171d95731378\") " pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:33.762734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.762699    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgrx5\" (UniqueName: \"kubernetes.io/projected/ef04ec97-4718-4f7b-9aae-171d95731378-kube-api-access-lgrx5\") pod \"test-curl-7470-140200320999760\" (UID: \"ef04ec97-4718-4f7b-9aae-171d95731378\") " pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:33.771907 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.771878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgrx5\" (UniqueName: \"kubernetes.io/projected/ef04ec97-4718-4f7b-9aae-171d95731378-kube-api-access-lgrx5\") pod \"test-curl-7470-140200320999760\" (UID: \"ef04ec97-4718-4f7b-9aae-171d95731378\") " pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:33.887121 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:33.887023    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:34.022457 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:34.022428    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:34.024236 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:29:34.024207    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef04ec97_4718_4f7b_9aae_171d95731378.slice/crio-7c749efc4345fb280f326d73f7014ec8e41b02000d06ed91dcb4186d81db3cc3 WatchSource:0}: Error finding container 7c749efc4345fb280f326d73f7014ec8e41b02000d06ed91dcb4186d81db3cc3: Status 404 returned error can't find the container with id 7c749efc4345fb280f326d73f7014ec8e41b02000d06ed91dcb4186d81db3cc3
Aug 04 12:29:34.025466 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:34.025445    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 12:29:34.993585 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:34.993493    2558 generic.go:356] "Generic (PLEG): container finished" podID="ef04ec97-4718-4f7b-9aae-171d95731378" containerID="af8debf535a1aa9fb847a7777eb49ce0838eba204d24bb89a26115421bf9935b" exitCode=0
Aug 04 12:29:34.994676 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:34.994653    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7470-140200320999760" event={"ID":"ef04ec97-4718-4f7b-9aae-171d95731378","Type":"ContainerDied","Data":"af8debf535a1aa9fb847a7777eb49ce0838eba204d24bb89a26115421bf9935b"}
Aug 04 12:29:34.994732 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:34.994686    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7470-140200320999760" event={"ID":"ef04ec97-4718-4f7b-9aae-171d95731378","Type":"ContainerStarted","Data":"7c749efc4345fb280f326d73f7014ec8e41b02000d06ed91dcb4186d81db3cc3"}
Aug 04 12:29:36.135823 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.135794    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:36.158859 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.158799    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7470-140200320999760_ef04ec97-4718-4f7b-9aae-171d95731378/test-curl-7470-140200320999760/0.log"
Aug 04 12:29:36.184111 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.184072    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgrx5\" (UniqueName: \"kubernetes.io/projected/ef04ec97-4718-4f7b-9aae-171d95731378-kube-api-access-lgrx5\") pod \"ef04ec97-4718-4f7b-9aae-171d95731378\" (UID: \"ef04ec97-4718-4f7b-9aae-171d95731378\") "
Aug 04 12:29:36.184536 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.184516    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:36.186074 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.186046    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef04ec97-4718-4f7b-9aae-171d95731378-kube-api-access-lgrx5" (OuterVolumeSpecName: "kube-api-access-lgrx5") pod "ef04ec97-4718-4f7b-9aae-171d95731378" (UID: "ef04ec97-4718-4f7b-9aae-171d95731378"). InnerVolumeSpecName "kube-api-access-lgrx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:29:36.186449 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.186433    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:36.284883 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.284859    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgrx5\" (UniqueName: \"kubernetes.io/projected/ef04ec97-4718-4f7b-9aae-171d95731378-kube-api-access-lgrx5\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:36.315155 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.315128    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:36.315531 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.315518    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef04ec97-4718-4f7b-9aae-171d95731378" containerName="test-curl-7470-140200320999760"
Aug 04 12:29:36.315580 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.315533    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef04ec97-4718-4f7b-9aae-171d95731378" containerName="test-curl-7470-140200320999760"
Aug 04 12:29:36.315620 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.315611    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef04ec97-4718-4f7b-9aae-171d95731378" containerName="test-curl-7470-140200320999760"
Aug 04 12:29:36.318646 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.318631    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:36.326405 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.326386    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:36.385723 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.385691    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nflh8\" (UniqueName: \"kubernetes.io/projected/7c41f3cd-14d0-4710-be4b-d572008a4fd5-kube-api-access-nflh8\") pod \"test-curl-7470-140200320999760\" (UID: \"7c41f3cd-14d0-4710-be4b-d572008a4fd5\") " pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:36.486701 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.486668    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nflh8\" (UniqueName: \"kubernetes.io/projected/7c41f3cd-14d0-4710-be4b-d572008a4fd5-kube-api-access-nflh8\") pod \"test-curl-7470-140200320999760\" (UID: \"7c41f3cd-14d0-4710-be4b-d572008a4fd5\") " pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:36.495404 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.495374    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nflh8\" (UniqueName: \"kubernetes.io/projected/7c41f3cd-14d0-4710-be4b-d572008a4fd5-kube-api-access-nflh8\") pod \"test-curl-7470-140200320999760\" (UID: \"7c41f3cd-14d0-4710-be4b-d572008a4fd5\") " pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:36.629427 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.629348    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:36.757838 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.757809    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:36.759264 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:29:36.759241    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c41f3cd_14d0_4710_be4b_d572008a4fd5.slice/crio-bb8e7a7d0a9c55eb89e82bac98f31caec81d1da824059d287b034c43c9797a3f WatchSource:0}: Error finding container bb8e7a7d0a9c55eb89e82bac98f31caec81d1da824059d287b034c43c9797a3f: Status 404 returned error can't find the container with id bb8e7a7d0a9c55eb89e82bac98f31caec81d1da824059d287b034c43c9797a3f
Aug 04 12:29:36.994908 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:36.994825    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef04ec97-4718-4f7b-9aae-171d95731378" path="/var/lib/kubelet/pods/ef04ec97-4718-4f7b-9aae-171d95731378/volumes"
Aug 04 12:29:37.004684 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:37.004658    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7470-140200320999760" event={"ID":"7c41f3cd-14d0-4710-be4b-d572008a4fd5","Type":"ContainerStarted","Data":"bb8e7a7d0a9c55eb89e82bac98f31caec81d1da824059d287b034c43c9797a3f"}
Aug 04 12:29:37.005729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:37.005712    2558 scope.go:117] "RemoveContainer" containerID="af8debf535a1aa9fb847a7777eb49ce0838eba204d24bb89a26115421bf9935b"
Aug 04 12:29:37.005729 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:37.005722    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:38.013171 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:38.013136    2558 generic.go:356] "Generic (PLEG): container finished" podID="7c41f3cd-14d0-4710-be4b-d572008a4fd5" containerID="3f703e3ae840aa63f40aaa5b7755d4e54686cc4819c8937a8f137fcdabf0c043" exitCode=0
Aug 04 12:29:38.013584 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:38.013187    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7470-140200320999760" event={"ID":"7c41f3cd-14d0-4710-be4b-d572008a4fd5","Type":"ContainerDied","Data":"3f703e3ae840aa63f40aaa5b7755d4e54686cc4819c8937a8f137fcdabf0c043"}
Aug 04 12:29:39.155571 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.155548    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:39.178644 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.178620    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7470-140200320999760_7c41f3cd-14d0-4710-be4b-d572008a4fd5/test-curl-7470-140200320999760/0.log"
Aug 04 12:29:39.202010 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.201972    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:39.205652 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.205630    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7470-140200320999760"]
Aug 04 12:29:39.210941 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.210917    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nflh8\" (UniqueName: \"kubernetes.io/projected/7c41f3cd-14d0-4710-be4b-d572008a4fd5-kube-api-access-nflh8\") pod \"7c41f3cd-14d0-4710-be4b-d572008a4fd5\" (UID: \"7c41f3cd-14d0-4710-be4b-d572008a4fd5\") "
Aug 04 12:29:39.212886 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.212860    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c41f3cd-14d0-4710-be4b-d572008a4fd5-kube-api-access-nflh8" (OuterVolumeSpecName: "kube-api-access-nflh8") pod "7c41f3cd-14d0-4710-be4b-d572008a4fd5" (UID: "7c41f3cd-14d0-4710-be4b-d572008a4fd5"). InnerVolumeSpecName "kube-api-access-nflh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:29:39.312604 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.312499    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nflh8\" (UniqueName: \"kubernetes.io/projected/7c41f3cd-14d0-4710-be4b-d572008a4fd5-kube-api-access-nflh8\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:29:39.737754 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:39.737674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:40.024171 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:40.024141    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb8e7a7d0a9c55eb89e82bac98f31caec81d1da824059d287b034c43c9797a3f"
Aug 04 12:29:40.024171 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:40.024147    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:40.034847 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:29:40.034820    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7470-140200320999760\" is forbidden: User \"system:node:ip-10-0-129-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-11.ec2.internal' and this object" podUID="7c41f3cd-14d0-4710-be4b-d572008a4fd5" pod="opendatahub/test-curl-7470-140200320999760"
Aug 04 12:29:40.134483 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:40.134446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:40.398421 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:40.398334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:40.994801 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:40.994771    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c41f3cd-14d0-4710-be4b-d572008a4fd5" path="/var/lib/kubelet/pods/7c41f3cd-14d0-4710-be4b-d572008a4fd5/volumes"
Aug 04 12:29:43.392440 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:43.392408    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:43.572076 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:43.572049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:44.476120 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.476089    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"]
Aug 04 12:29:44.476534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.476503    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c41f3cd-14d0-4710-be4b-d572008a4fd5" containerName="test-curl-7470-140200320999760"
Aug 04 12:29:44.476534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.476516    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c41f3cd-14d0-4710-be4b-d572008a4fd5" containerName="test-curl-7470-140200320999760"
Aug 04 12:29:44.476619 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.476576    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c41f3cd-14d0-4710-be4b-d572008a4fd5" containerName="test-curl-7470-140200320999760"
Aug 04 12:29:44.479965 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.479949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.484414 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.484392    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a888a4-serving-cert\""
Aug 04 12:29:44.505989 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.505969    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"]
Aug 04 12:29:44.507692 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.507670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:44.557013 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.556988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p8wj\" (UniqueName: \"kubernetes.io/projected/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-kube-api-access-2p8wj\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.557152 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.557052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.557152 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.557132    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.658471 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.658438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.658638 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.658485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p8wj\" (UniqueName: \"kubernetes.io/projected/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-kube-api-access-2p8wj\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.658638 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.658542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.658716 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:29:44.658651    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-serving-cert: secret "maas-api-e2e-ipp-a-a888a4-serving-cert" not found
Aug 04 12:29:44.658716 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:29:44.658710    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls podName:f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3 nodeName:}" failed. No retries permitted until 2026-08-04 12:29:45.15869071 +0000 UTC m=+3398.729603494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls") pod "maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" (UID: "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3") : secret "maas-api-e2e-ipp-a-a888a4-serving-cert" not found
Aug 04 12:29:44.659068 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.659047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.669934 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.669903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p8wj\" (UniqueName: \"kubernetes.io/projected/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-kube-api-access-2p8wj\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:44.760095 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:44.760066    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:45.163772 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:45.163691    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:45.165868 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:45.165852    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:45.171219 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:45.171191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:45.267629 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:45.267600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:45.389909 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:45.389877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:45.544317 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:45.544294    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"]
Aug 04 12:29:45.546175 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:29:45.546148    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f2b7cf_4e3c_4e23_9f39_3a666aebeae3.slice/crio-2da70f48014d68682a2e028eea39e0fd0c8c26cbe14d333b4fbeddfc9101221a WatchSource:0}: Error finding container 2da70f48014d68682a2e028eea39e0fd0c8c26cbe14d333b4fbeddfc9101221a: Status 404 returned error can't find the container with id 2da70f48014d68682a2e028eea39e0fd0c8c26cbe14d333b4fbeddfc9101221a
Aug 04 12:29:46.055218 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:46.055175    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" event={"ID":"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3","Type":"ContainerStarted","Data":"2da70f48014d68682a2e028eea39e0fd0c8c26cbe14d333b4fbeddfc9101221a"}
Aug 04 12:29:47.062109 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:47.062070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" event={"ID":"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3","Type":"ContainerStarted","Data":"b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13"}
Aug 04 12:29:47.062526 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:47.062131    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:29:47.102360 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:47.102314    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" podStartSLOduration=2.147523741 podStartE2EDuration="3.102298443s" totalImagesPullingTime="954.774702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:29:47.101370609 +0000 UTC m=+3400.672283413" watchObservedRunningTime="2026-08-04 12:29:47.102298443 +0000 UTC m=+3400.673211246"
Aug 04 12:29:48.015979 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:48.015947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:48.104386 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:48.104354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:29:53.072253 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:29:53.072216    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:30:04.303486 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:04.303405    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"]
Aug 04 12:30:04.305321 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:04.305301    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764065-56c7n"]
Aug 04 12:30:04.994228 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:04.994189    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d22ade-8a2e-4556-9a43-893b4d89a49c" path="/var/lib/kubelet/pods/50d22ade-8a2e-4556-9a43-893b4d89a49c/volumes"
Aug 04 12:30:07.481398 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:07.481366    2558 scope.go:117] "RemoveContainer" containerID="e38d018de62c519a221d285a7f709c88abca650410c8f3a99cd6112b3f72335d"
Aug 04 12:30:08.515862 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:08.515792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:09.277734 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:09.277698    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:09.742439 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:09.742346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:10.419010 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:10.418982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:12.718334 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:12.718298    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:12.819248 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:12.819218    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:14.176832 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.176798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:14.221359 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.221326    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"]
Aug 04 12:30:14.227550 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.227528    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.230450 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.230429    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0778c1-serving-cert\""
Aug 04 12:30:14.324033 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.324002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ae6842-48f2-4700-bf87-b557cff0a55f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.324213 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.324064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.324213 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.324183    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8wkr\" (UniqueName: \"kubernetes.io/projected/06ae6842-48f2-4700-bf87-b557cff0a55f-kube-api-access-j8wkr\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.425403 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.425373    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ae6842-48f2-4700-bf87-b557cff0a55f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.425585 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.425425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.425585 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.425485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8wkr\" (UniqueName: \"kubernetes.io/projected/06ae6842-48f2-4700-bf87-b557cff0a55f-kube-api-access-j8wkr\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.426035 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.426011    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ae6842-48f2-4700-bf87-b557cff0a55f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.427954 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.427884    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.434550 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.434528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:14.437677 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.437649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8wkr\" (UniqueName: \"kubernetes.io/projected/06ae6842-48f2-4700-bf87-b557cff0a55f-kube-api-access-j8wkr\") pod \"maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.537757 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.537722    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:14.889364 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:14.889326    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"]
Aug 04 12:30:15.088023 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:15.087991    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"]
Aug 04 12:30:15.089525 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:30:15.089503    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ae6842_48f2_4700_bf87_b557cff0a55f.slice/crio-defd8bdd991707a9bfa11808d78fc23005353e6c450317d7a351d2125409238d WatchSource:0}: Error finding container defd8bdd991707a9bfa11808d78fc23005353e6c450317d7a351d2125409238d: Status 404 returned error can't find the container with id defd8bdd991707a9bfa11808d78fc23005353e6c450317d7a351d2125409238d
Aug 04 12:30:15.203743 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:15.203657    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" event={"ID":"06ae6842-48f2-4700-bf87-b557cff0a55f","Type":"ContainerStarted","Data":"defd8bdd991707a9bfa11808d78fc23005353e6c450317d7a351d2125409238d"}
Aug 04 12:30:16.210571 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:16.210536    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" event={"ID":"06ae6842-48f2-4700-bf87-b557cff0a55f","Type":"ContainerStarted","Data":"b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8"}
Aug 04 12:30:16.210991 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:16.210626    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:16.231836 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:16.231788    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" podStartSLOduration=1.927209983 podStartE2EDuration="2.231772502s" totalImagesPullingTime="304.562519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:16.229267259 +0000 UTC m=+3429.800180063" watchObservedRunningTime="2026-08-04 12:30:16.231772502 +0000 UTC m=+3429.802685306"
Aug 04 12:30:22.223294 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:22.223255    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:30:37.749978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:37.749947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:30:52.320449 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:30:52.320410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:31:03.977344 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.977310    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"]
Aug 04 12:31:03.981087 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.981065    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:03.985638 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.985618    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-a888a4\"/\"openshift-service-ca.crt\""
Aug 04 12:31:03.985744 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.985620    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-a888a4\"/\"kube-root-ca.crt\""
Aug 04 12:31:03.985744 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.985618    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-a888a4\"/\"default-dockercfg-jksfk\""
Aug 04 12:31:03.985857 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.985625    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-a888a4\"/\"ipp-route-a888a4-kserve-self-signed-certs\""
Aug 04 12:31:03.994042 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:03.994018    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"]
Aug 04 12:31:04.008883 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.008849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:31:04.075514 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.075482    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-dshm\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.075514 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.075516    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tls-certs\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.075738 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.075542    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-home\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.075738 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.075625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfcc6\" (UniqueName: \"kubernetes.io/projected/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-kube-api-access-cfcc6\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.075811 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.075749    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tmp-dir\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.075811 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.075780    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-model-cache\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176473 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-dshm\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176473 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tls-certs\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176700 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-home\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176700 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfcc6\" (UniqueName: \"kubernetes.io/projected/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-kube-api-access-cfcc6\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176700 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176580    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tmp-dir\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176700 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-model-cache\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.176946 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-home\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.177031 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.176957    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tmp-dir\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.177084 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.177026    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-model-cache\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.178916 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.178886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-dshm\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.179185 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.179166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tls-certs\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.185561 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.185539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfcc6\" (UniqueName: \"kubernetes.io/projected/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-kube-api-access-cfcc6\") pod \"ipp-route-a888a4-kserve-d6cd69f5-47dmh\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") " pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.290786 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.290751    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:04.429469 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.429441    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"]
Aug 04 12:31:04.430953 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:31:04.430921    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefe2f123_c0eb_4f94_ac85_fdea33aa6e29.slice/crio-075ffbd58a464929a6fe24163c4428a1bdda55423a7bb7357706a45bfc5b1c73 WatchSource:0}: Error finding container 075ffbd58a464929a6fe24163c4428a1bdda55423a7bb7357706a45bfc5b1c73: Status 404 returned error can't find the container with id 075ffbd58a464929a6fe24163c4428a1bdda55423a7bb7357706a45bfc5b1c73
Aug 04 12:31:04.461873 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:04.461850    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh" event={"ID":"efe2f123-c0eb-4f94-ac85-fdea33aa6e29","Type":"ContainerStarted","Data":"075ffbd58a464929a6fe24163c4428a1bdda55423a7bb7357706a45bfc5b1c73"}
Aug 04 12:31:05.468955 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:05.468921    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh" event={"ID":"efe2f123-c0eb-4f94-ac85-fdea33aa6e29","Type":"ContainerStarted","Data":"95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446"}
Aug 04 12:31:05.491178 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:05.491132    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh" podStartSLOduration=2.491116598 podStartE2EDuration="2.491116598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:31:05.48939371 +0000 UTC m=+3479.060306528" watchObservedRunningTime="2026-08-04 12:31:05.491116598 +0000 UTC m=+3479.062029401"
Aug 04 12:31:14.291463 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:14.291429    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:14.291977 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:14.291488    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:14.309669 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:14.309639    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:31:14.528694 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:31:14.528667    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:32:01.036813 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:01.036775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:01.359119 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:01.359023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:01.603264 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:01.603233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:04.067804 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:04.067771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:04.167648 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:04.167613    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:05.158983 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.158951    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"]
Aug 04 12:32:05.163145 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.163125    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.166173 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.166153    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-672896-serving-cert\""
Aug 04 12:32:05.176847 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.176822    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"]
Aug 04 12:32:05.202168 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.202136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:05.215110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.215080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90278e1-74d1-4472-be67-69e5a872569b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.215251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.215120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkbxn\" (UniqueName: \"kubernetes.io/projected/a90278e1-74d1-4472-be67-69e5a872569b-kube-api-access-pkbxn\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.215251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.215221    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90278e1-74d1-4472-be67-69e5a872569b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.316110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.316075    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90278e1-74d1-4472-be67-69e5a872569b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.316110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.316118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkbxn\" (UniqueName: \"kubernetes.io/projected/a90278e1-74d1-4472-be67-69e5a872569b-kube-api-access-pkbxn\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.316376 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.316166    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90278e1-74d1-4472-be67-69e5a872569b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.316834 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.316800    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90278e1-74d1-4472-be67-69e5a872569b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.318690 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.318669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90278e1-74d1-4472-be67-69e5a872569b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.326038 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.326014    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkbxn\" (UniqueName: \"kubernetes.io/projected/a90278e1-74d1-4472-be67-69e5a872569b-kube-api-access-pkbxn\") pod \"maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.358704 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.358672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:05.476627 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.476547    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:05.616760 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.616733    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"]
Aug 04 12:32:05.619010 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:32:05.618981    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda90278e1_74d1_4472_be67_69e5a872569b.slice/crio-18c46ad28fcf85ba00f1affbd62ab6d49f3cb9696f376ee6860d00c43ca9e069 WatchSource:0}: Error finding container 18c46ad28fcf85ba00f1affbd62ab6d49f3cb9696f376ee6860d00c43ca9e069: Status 404 returned error can't find the container with id 18c46ad28fcf85ba00f1affbd62ab6d49f3cb9696f376ee6860d00c43ca9e069
Aug 04 12:32:05.705094 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.705057    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:05.783289 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.783243    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng" event={"ID":"a90278e1-74d1-4472-be67-69e5a872569b","Type":"ContainerStarted","Data":"18c46ad28fcf85ba00f1affbd62ab6d49f3cb9696f376ee6860d00c43ca9e069"}
Aug 04 12:32:05.808037 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:05.808007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:06.790014 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:06.789975    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng" event={"ID":"a90278e1-74d1-4472-be67-69e5a872569b","Type":"ContainerStarted","Data":"f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958"}
Aug 04 12:32:06.790518 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:06.790045    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:06.812702 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:06.812653    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng" podStartSLOduration=1.3438679869999999 podStartE2EDuration="1.812636485s" totalImagesPullingTime="468.768498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:32:06.810638845 +0000 UTC m=+3540.381551648" watchObservedRunningTime="2026-08-04 12:32:06.812636485 +0000 UTC m=+3540.383549328"
Aug 04 12:32:12.802018 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:12.801987    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:29.048330 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:29.048268    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:35.936850 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:35.936811    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"]
Aug 04 12:32:35.937388 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:35.937061    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng" podUID="a90278e1-74d1-4472-be67-69e5a872569b" containerName="maas-api" containerID="cri-o://f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958" gracePeriod=30
Aug 04 12:32:35.992681 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:35.992626    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:36.092204 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.092166    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:36.177446 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.177416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:36.202841 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.202781    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:36.317674 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.317640    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90278e1-74d1-4472-be67-69e5a872569b-openshift-service-ca\") pod \"a90278e1-74d1-4472-be67-69e5a872569b\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") "
Aug 04 12:32:36.317845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.317781    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkbxn\" (UniqueName: \"kubernetes.io/projected/a90278e1-74d1-4472-be67-69e5a872569b-kube-api-access-pkbxn\") pod \"a90278e1-74d1-4472-be67-69e5a872569b\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") "
Aug 04 12:32:36.317845 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.317816    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90278e1-74d1-4472-be67-69e5a872569b-maas-api-tls\") pod \"a90278e1-74d1-4472-be67-69e5a872569b\" (UID: \"a90278e1-74d1-4472-be67-69e5a872569b\") "
Aug 04 12:32:36.318029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.318006    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a90278e1-74d1-4472-be67-69e5a872569b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a90278e1-74d1-4472-be67-69e5a872569b" (UID: "a90278e1-74d1-4472-be67-69e5a872569b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:32:36.318148 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.318133    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90278e1-74d1-4472-be67-69e5a872569b-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:32:36.319863 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.319840    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a90278e1-74d1-4472-be67-69e5a872569b-kube-api-access-pkbxn" (OuterVolumeSpecName: "kube-api-access-pkbxn") pod "a90278e1-74d1-4472-be67-69e5a872569b" (UID: "a90278e1-74d1-4472-be67-69e5a872569b"). InnerVolumeSpecName "kube-api-access-pkbxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:32:36.320004 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.319983    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a90278e1-74d1-4472-be67-69e5a872569b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a90278e1-74d1-4472-be67-69e5a872569b" (UID: "a90278e1-74d1-4472-be67-69e5a872569b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:32:36.418875 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.418837    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkbxn\" (UniqueName: \"kubernetes.io/projected/a90278e1-74d1-4472-be67-69e5a872569b-kube-api-access-pkbxn\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:32:36.418875 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.418872    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90278e1-74d1-4472-be67-69e5a872569b-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:32:36.608140 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.608111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:36.953507 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.953423    2558 generic.go:356] "Generic (PLEG): container finished" podID="a90278e1-74d1-4472-be67-69e5a872569b" containerID="f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958" exitCode=0
Aug 04 12:32:36.953507 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.953465    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"
Aug 04 12:32:36.953978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.953464    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng" event={"ID":"a90278e1-74d1-4472-be67-69e5a872569b","Type":"ContainerDied","Data":"f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958"}
Aug 04 12:32:36.953978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.953599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng" event={"ID":"a90278e1-74d1-4472-be67-69e5a872569b","Type":"ContainerDied","Data":"18c46ad28fcf85ba00f1affbd62ab6d49f3cb9696f376ee6860d00c43ca9e069"}
Aug 04 12:32:36.953978 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.953630    2558 scope.go:117] "RemoveContainer" containerID="f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958"
Aug 04 12:32:36.963870 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.963851    2558 scope.go:117] "RemoveContainer" containerID="f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958"
Aug 04 12:32:36.964105 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:32:36.964087    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958\": container with ID starting with f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958 not found: ID does not exist" containerID="f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958"
Aug 04 12:32:36.964148 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.964116    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958"} err="failed to get container status \"f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958\": rpc error: code = NotFound desc = could not find container \"f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958\": container with ID starting with f143352432099ad52de41830d75d5496e27969923f9d53c0cedd41f2264e3958 not found: ID does not exist"
Aug 04 12:32:36.987140 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.987117    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"]
Aug 04 12:32:36.994689 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:36.994663    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-672896-59584d595c-2xtng"]
Aug 04 12:32:38.994894 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:38.994858    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a90278e1-74d1-4472-be67-69e5a872569b" path="/var/lib/kubelet/pods/a90278e1-74d1-4472-be67-69e5a872569b/volumes"
Aug 04 12:32:43.557637 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:43.557606    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:43.672252 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:43.672211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:43.822689 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:43.822594    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:32:44.043361 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:32:44.043329    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:03.041607 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.041512    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"]
Aug 04 12:33:03.042102 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.041761    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" podUID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" containerName="maas-api" containerID="cri-o://b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13" gracePeriod=30
Aug 04 12:33:03.068142 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.068104    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" podUID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.88:8443/health\": dial tcp 10.132.0.88:8443: connect: connection refused"
Aug 04 12:33:03.087834 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.087808    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:03.187688 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.187649    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:03.289218 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.289191    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:33:03.290596 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.290570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:03.367254 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.367161    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p8wj\" (UniqueName: \"kubernetes.io/projected/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-kube-api-access-2p8wj\") pod \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") "
Aug 04 12:33:03.367254 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.367222    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls\") pod \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") "
Aug 04 12:33:03.367534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.367268    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-openshift-service-ca\") pod \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\" (UID: \"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3\") "
Aug 04 12:33:03.367626 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.367606    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" (UID: "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:33:03.367711 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.367699    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:03.369377 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.369350    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-kube-api-access-2p8wj" (OuterVolumeSpecName: "kube-api-access-2p8wj") pod "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" (UID: "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3"). InnerVolumeSpecName "kube-api-access-2p8wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:33:03.369490 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.369461    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" (UID: "f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:33:03.399417 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.399387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:03.469096 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.469066    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p8wj\" (UniqueName: \"kubernetes.io/projected/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-kube-api-access-2p8wj\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:03.469096 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.469097    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:03.755443 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:03.755393    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:04.103760 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.103672    2558 generic.go:356] "Generic (PLEG): container finished" podID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" containerID="b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13" exitCode=0
Aug 04 12:33:04.103760 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.103716    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"
Aug 04 12:33:04.104296 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.103763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" event={"ID":"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3","Type":"ContainerDied","Data":"b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13"}
Aug 04 12:33:04.104296 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.103818    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s" event={"ID":"f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3","Type":"ContainerDied","Data":"2da70f48014d68682a2e028eea39e0fd0c8c26cbe14d333b4fbeddfc9101221a"}
Aug 04 12:33:04.104296 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.103850    2558 scope.go:117] "RemoveContainer" containerID="b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13"
Aug 04 12:33:04.113449 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.113433    2558 scope.go:117] "RemoveContainer" containerID="b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13"
Aug 04 12:33:04.113698 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:33:04.113680    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13\": container with ID starting with b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13 not found: ID does not exist" containerID="b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13"
Aug 04 12:33:04.113780 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.113703    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13"} err="failed to get container status \"b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13\": rpc error: code = NotFound desc = could not find container \"b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13\": container with ID starting with b8ce05b9c0d7decbdbfe9e2f4da03914c6cc0e5521c9a42c4886060a0a9a9a13 not found: ID does not exist"
Aug 04 12:33:04.131152 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.131128    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"]
Aug 04 12:33:04.134769 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:04.134745    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a888a4-6586b8c686-rjr8s"]
Aug 04 12:33:05.001226 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:05.001148    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" path="/var/lib/kubelet/pods/f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3/volumes"
Aug 04 12:33:07.230151 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:07.230124    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:33:07.239248 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:07.239228    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:33:08.708028 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:08.707991    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"]
Aug 04 12:33:08.754455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:08.754418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:23.727525 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:23.727476    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh" podUID="efe2f123-c0eb-4f94-ac85-fdea33aa6e29" containerName="main" containerID="cri-o://95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446" gracePeriod=45
Aug 04 12:33:23.995814 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:23.995790    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:33:24.168149 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168115    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-home\") pod \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") "
Aug 04 12:33:24.168363 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168165    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tls-certs\") pod \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") "
Aug 04 12:33:24.168363 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168201    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfcc6\" (UniqueName: \"kubernetes.io/projected/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-kube-api-access-cfcc6\") pod \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") "
Aug 04 12:33:24.168363 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168239    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-model-cache\") pod \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") "
Aug 04 12:33:24.168363 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168309    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-dshm\") pod \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") "
Aug 04 12:33:24.168363 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168347    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tmp-dir\") pod \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\" (UID: \"efe2f123-c0eb-4f94-ac85-fdea33aa6e29\") "
Aug 04 12:33:24.168653 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168432    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-home" (OuterVolumeSpecName: "home") pod "efe2f123-c0eb-4f94-ac85-fdea33aa6e29" (UID: "efe2f123-c0eb-4f94-ac85-fdea33aa6e29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:33:24.168653 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168583    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-model-cache" (OuterVolumeSpecName: "model-cache") pod "efe2f123-c0eb-4f94-ac85-fdea33aa6e29" (UID: "efe2f123-c0eb-4f94-ac85-fdea33aa6e29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:33:24.168773 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "efe2f123-c0eb-4f94-ac85-fdea33aa6e29" (UID: "efe2f123-c0eb-4f94-ac85-fdea33aa6e29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:33:24.168825 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168792    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tmp-dir\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:24.168825 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168812    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-home\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:24.168919 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.168827    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-model-cache\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:24.170695 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.170667    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-kube-api-access-cfcc6" (OuterVolumeSpecName: "kube-api-access-cfcc6") pod "efe2f123-c0eb-4f94-ac85-fdea33aa6e29" (UID: "efe2f123-c0eb-4f94-ac85-fdea33aa6e29"). InnerVolumeSpecName "kube-api-access-cfcc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:33:24.170895 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.170870    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "efe2f123-c0eb-4f94-ac85-fdea33aa6e29" (UID: "efe2f123-c0eb-4f94-ac85-fdea33aa6e29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:33:24.170961 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.170939    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-dshm" (OuterVolumeSpecName: "dshm") pod "efe2f123-c0eb-4f94-ac85-fdea33aa6e29" (UID: "efe2f123-c0eb-4f94-ac85-fdea33aa6e29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 12:33:24.213920 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.213881    2558 generic.go:356] "Generic (PLEG): container finished" podID="efe2f123-c0eb-4f94-ac85-fdea33aa6e29" containerID="95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446" exitCode=0
Aug 04 12:33:24.214110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.213927    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"
Aug 04 12:33:24.214110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.213937    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh" event={"ID":"efe2f123-c0eb-4f94-ac85-fdea33aa6e29","Type":"ContainerDied","Data":"95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446"}
Aug 04 12:33:24.214110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.213977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh" event={"ID":"efe2f123-c0eb-4f94-ac85-fdea33aa6e29","Type":"ContainerDied","Data":"075ffbd58a464929a6fe24163c4428a1bdda55423a7bb7357706a45bfc5b1c73"}
Aug 04 12:33:24.214110 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.213998    2558 scope.go:117] "RemoveContainer" containerID="95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446"
Aug 04 12:33:24.223751 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.223732    2558 scope.go:117] "RemoveContainer" containerID="95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446"
Aug 04 12:33:24.224049 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:33:24.224024    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446\": container with ID starting with 95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446 not found: ID does not exist" containerID="95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446"
Aug 04 12:33:24.224092 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.224060    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446"} err="failed to get container status \"95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446\": rpc error: code = NotFound desc = could not find container \"95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446\": container with ID starting with 95a394ba614900f0adfc8c5d26544e7294d3c8c33897041ac0fd3b7fbe653446 not found: ID does not exist"
Aug 04 12:33:24.237630 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.237556    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"]
Aug 04 12:33:24.247198 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.247169    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-a888a4/ipp-route-a888a4-kserve-d6cd69f5-47dmh"]
Aug 04 12:33:24.269742 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.269718    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-tls-certs\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:24.269742 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.269742    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfcc6\" (UniqueName: \"kubernetes.io/projected/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-kube-api-access-cfcc6\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:24.269909 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.269751    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe2f123-c0eb-4f94-ac85-fdea33aa6e29-dshm\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:33:24.994637 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:24.994606    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe2f123-c0eb-4f94-ac85-fdea33aa6e29" path="/var/lib/kubelet/pods/efe2f123-c0eb-4f94-ac85-fdea33aa6e29/volumes"
Aug 04 12:33:44.966556 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:44.966520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:46.377310 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:46.377233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:52.011395 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:52.011359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:52.073663 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:52.073597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:52.633441 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:52.633402    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:33:52.706992 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:33:52.706949    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:03.018453 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.018422    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"]
Aug 04 12:34:03.046400 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.046372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:03.097199 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:03.097161    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-serving-cert: secret "maas-api-e2e-ipp-b-0778c1-serving-cert" not found
Aug 04 12:34:03.097374 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:03.097259    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls podName:06ae6842-48f2-4700-bf87-b557cff0a55f nodeName:}" failed. No retries permitted until 2026-08-04 12:34:03.597235447 +0000 UTC m=+3657.168148244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls") pod "maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" (UID: "06ae6842-48f2-4700-bf87-b557cff0a55f") : secret "maas-api-e2e-ipp-b-0778c1-serving-cert" not found
Aug 04 12:34:03.147420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.147382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:03.220553 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.220502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:03.420420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.420329    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" podUID="06ae6842-48f2-4700-bf87-b557cff0a55f" containerName="maas-api" containerID="cri-o://b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8" gracePeriod=30
Aug 04 12:34:03.584593 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.584557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:03.601987 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:03.601960    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-serving-cert: secret "maas-api-e2e-ipp-b-0778c1-serving-cert" not found
Aug 04 12:34:03.602133 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:03.602033    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls podName:06ae6842-48f2-4700-bf87-b557cff0a55f nodeName:}" failed. No retries permitted until 2026-08-04 12:34:04.602019544 +0000 UTC m=+3658.172932326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls") pod "maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" (UID: "06ae6842-48f2-4700-bf87-b557cff0a55f") : secret "maas-api-e2e-ipp-b-0778c1-serving-cert" not found
Aug 04 12:34:03.666780 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.666754    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:34:03.803767 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.803736    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls\") pod \"06ae6842-48f2-4700-bf87-b557cff0a55f\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") "
Aug 04 12:34:03.803939 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.803779    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8wkr\" (UniqueName: \"kubernetes.io/projected/06ae6842-48f2-4700-bf87-b557cff0a55f-kube-api-access-j8wkr\") pod \"06ae6842-48f2-4700-bf87-b557cff0a55f\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") "
Aug 04 12:34:03.803939 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.803882    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ae6842-48f2-4700-bf87-b557cff0a55f-openshift-service-ca\") pod \"06ae6842-48f2-4700-bf87-b557cff0a55f\" (UID: \"06ae6842-48f2-4700-bf87-b557cff0a55f\") "
Aug 04 12:34:03.804251 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.804228    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06ae6842-48f2-4700-bf87-b557cff0a55f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06ae6842-48f2-4700-bf87-b557cff0a55f" (UID: "06ae6842-48f2-4700-bf87-b557cff0a55f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:34:03.805912 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.805888    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ae6842-48f2-4700-bf87-b557cff0a55f-kube-api-access-j8wkr" (OuterVolumeSpecName: "kube-api-access-j8wkr") pod "06ae6842-48f2-4700-bf87-b557cff0a55f" (UID: "06ae6842-48f2-4700-bf87-b557cff0a55f"). InnerVolumeSpecName "kube-api-access-j8wkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:34:03.806012 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.805934    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06ae6842-48f2-4700-bf87-b557cff0a55f" (UID: "06ae6842-48f2-4700-bf87-b557cff0a55f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:34:03.905420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.905392    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ae6842-48f2-4700-bf87-b557cff0a55f-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:03.905420 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.905420    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8wkr\" (UniqueName: \"kubernetes.io/projected/06ae6842-48f2-4700-bf87-b557cff0a55f-kube-api-access-j8wkr\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:03.905596 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:03.905430    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ae6842-48f2-4700-bf87-b557cff0a55f-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:04.427382 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.427343    2558 generic.go:356] "Generic (PLEG): container finished" podID="06ae6842-48f2-4700-bf87-b557cff0a55f" containerID="b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8" exitCode=0
Aug 04 12:34:04.427847 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.427397    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"
Aug 04 12:34:04.427847 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.427427    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" event={"ID":"06ae6842-48f2-4700-bf87-b557cff0a55f","Type":"ContainerDied","Data":"b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8"}
Aug 04 12:34:04.427847 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.427482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8" event={"ID":"06ae6842-48f2-4700-bf87-b557cff0a55f","Type":"ContainerDied","Data":"defd8bdd991707a9bfa11808d78fc23005353e6c450317d7a351d2125409238d"}
Aug 04 12:34:04.427847 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.427503    2558 scope.go:117] "RemoveContainer" containerID="b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8"
Aug 04 12:34:04.437459 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.437436    2558 scope.go:117] "RemoveContainer" containerID="b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8"
Aug 04 12:34:04.437721 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:04.437701    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8\": container with ID starting with b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8 not found: ID does not exist" containerID="b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8"
Aug 04 12:34:04.437780 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.437730    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8"} err="failed to get container status \"b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8\": rpc error: code = NotFound desc = could not find container \"b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8\": container with ID starting with b4323896b571d56cbfc1da1d96b0d6955b1582755b7e0d183fec32db081775c8 not found: ID does not exist"
Aug 04 12:34:04.451660 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.451638    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"]
Aug 04 12:34:04.457525 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.457502    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0778c1-69b55c78c9-jc9d8"]
Aug 04 12:34:04.993875 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:04.993843    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06ae6842-48f2-4700-bf87-b557cff0a55f" path="/var/lib/kubelet/pods/06ae6842-48f2-4700-bf87-b557cff0a55f/volumes"
Aug 04 12:34:09.735042 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:09.735008    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:09.835755 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:09.835719    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:09.993774 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:09.993700    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:10.208664 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:10.208638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:20.776921 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.776882    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"]
Aug 04 12:34:20.777526 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.777098    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr" podUID="69a7a597-d783-456f-a35d-14cedc4c4ab9" containerName="maas-api" containerID="cri-o://dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013" gracePeriod=30
Aug 04 12:34:20.804036 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.804009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:20.828833 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.828806    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"]
Aug 04 12:34:20.829029 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.829006    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv" podUID="6251a9a3-faec-4cd0-845e-e9ee3eee82b8" containerName="payload-processing" containerID="cri-o://0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12" gracePeriod=30
Aug 04 12:34:20.902676 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.902646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:20.981498 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:20.981451    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:21.060858 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.060825    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:21.087825 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.087802    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:34:21.113605 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.113574    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:34:21.158910 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.158879    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7a597-d783-456f-a35d-14cedc4c4ab9-maas-api-tls\") pod \"69a7a597-d783-456f-a35d-14cedc4c4ab9\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") "
Aug 04 12:34:21.159073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.158935    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-plugins-config-volume\") pod \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") "
Aug 04 12:34:21.159073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.159021    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7a597-d783-456f-a35d-14cedc4c4ab9-openshift-service-ca\") pod \"69a7a597-d783-456f-a35d-14cedc4c4ab9\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") "
Aug 04 12:34:21.159073 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.159058    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrl2n\" (UniqueName: \"kubernetes.io/projected/69a7a597-d783-456f-a35d-14cedc4c4ab9-kube-api-access-nrl2n\") pod \"69a7a597-d783-456f-a35d-14cedc4c4ab9\" (UID: \"69a7a597-d783-456f-a35d-14cedc4c4ab9\") "
Aug 04 12:34:21.159249 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.159102    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz82z\" (UniqueName: \"kubernetes.io/projected/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-kube-api-access-wz82z\") pod \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\" (UID: \"6251a9a3-faec-4cd0-845e-e9ee3eee82b8\") "
Aug 04 12:34:21.159431 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.159404    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6251a9a3-faec-4cd0-845e-e9ee3eee82b8" (UID: "6251a9a3-faec-4cd0-845e-e9ee3eee82b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:34:21.159493 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.159421    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a7a597-d783-456f-a35d-14cedc4c4ab9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69a7a597-d783-456f-a35d-14cedc4c4ab9" (UID: "69a7a597-d783-456f-a35d-14cedc4c4ab9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:34:21.161006 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.160987    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69a7a597-d783-456f-a35d-14cedc4c4ab9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69a7a597-d783-456f-a35d-14cedc4c4ab9" (UID: "69a7a597-d783-456f-a35d-14cedc4c4ab9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:34:21.161156 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.161136    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a7a597-d783-456f-a35d-14cedc4c4ab9-kube-api-access-nrl2n" (OuterVolumeSpecName: "kube-api-access-nrl2n") pod "69a7a597-d783-456f-a35d-14cedc4c4ab9" (UID: "69a7a597-d783-456f-a35d-14cedc4c4ab9"). InnerVolumeSpecName "kube-api-access-nrl2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:34:21.161209 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.161168    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-kube-api-access-wz82z" (OuterVolumeSpecName: "kube-api-access-wz82z") pod "6251a9a3-faec-4cd0-845e-e9ee3eee82b8" (UID: "6251a9a3-faec-4cd0-845e-e9ee3eee82b8"). InnerVolumeSpecName "kube-api-access-wz82z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:34:21.259894 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.259868    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7a597-d783-456f-a35d-14cedc4c4ab9-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:21.259894 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.259892    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrl2n\" (UniqueName: \"kubernetes.io/projected/69a7a597-d783-456f-a35d-14cedc4c4ab9-kube-api-access-nrl2n\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:21.260095 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.259902    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz82z\" (UniqueName: \"kubernetes.io/projected/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-kube-api-access-wz82z\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:21.260095 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.259911    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7a597-d783-456f-a35d-14cedc4c4ab9-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:21.260095 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.259920    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6251a9a3-faec-4cd0-845e-e9ee3eee82b8-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:21.367986 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.367908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:21.515543 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.515511    2558 generic.go:356] "Generic (PLEG): container finished" podID="6251a9a3-faec-4cd0-845e-e9ee3eee82b8" containerID="0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12" exitCode=0
Aug 04 12:34:21.515740 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.515565    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"
Aug 04 12:34:21.515740 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.515599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv" event={"ID":"6251a9a3-faec-4cd0-845e-e9ee3eee82b8","Type":"ContainerDied","Data":"0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12"}
Aug 04 12:34:21.515740 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.515650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv" event={"ID":"6251a9a3-faec-4cd0-845e-e9ee3eee82b8","Type":"ContainerDied","Data":"204727c12afbcd0828bcd465b053ef6bb4af417e06c499796800a3e88d5f8bb5"}
Aug 04 12:34:21.515740 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.515674    2558 scope.go:117] "RemoveContainer" containerID="0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12"
Aug 04 12:34:21.516952 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.516928    2558 generic.go:356] "Generic (PLEG): container finished" podID="69a7a597-d783-456f-a35d-14cedc4c4ab9" containerID="dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013" exitCode=0
Aug 04 12:34:21.517080 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.516966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr" event={"ID":"69a7a597-d783-456f-a35d-14cedc4c4ab9","Type":"ContainerDied","Data":"dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013"}
Aug 04 12:34:21.517080 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.517009    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr" event={"ID":"69a7a597-d783-456f-a35d-14cedc4c4ab9","Type":"ContainerDied","Data":"437c41908473f54d6169cf6814124e5f35e4ba3c13ccb60da1433e078c0422bd"}
Aug 04 12:34:21.517080 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.516970    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"
Aug 04 12:34:21.524998 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.524982    2558 scope.go:117] "RemoveContainer" containerID="0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12"
Aug 04 12:34:21.525265 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:21.525248    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12\": container with ID starting with 0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12 not found: ID does not exist" containerID="0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12"
Aug 04 12:34:21.525370 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.525294    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12"} err="failed to get container status \"0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12\": rpc error: code = NotFound desc = could not find container \"0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12\": container with ID starting with 0e59baa9579363116d204929f0241fa8f25e40a322d52317f597e97f1f127b12 not found: ID does not exist"
Aug 04 12:34:21.525370 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.525318    2558 scope.go:117] "RemoveContainer" containerID="dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013"
Aug 04 12:34:21.533890 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.533872    2558 scope.go:117] "RemoveContainer" containerID="dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013"
Aug 04 12:34:21.534121 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:21.534106    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013\": container with ID starting with dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013 not found: ID does not exist" containerID="dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013"
Aug 04 12:34:21.534162 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.534128    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013"} err="failed to get container status \"dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013\": rpc error: code = NotFound desc = could not find container \"dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013\": container with ID starting with dd04d79bf5819b49eb3b355b9ad93630182c0e35cc559e7747be13adf3f45013 not found: ID does not exist"
Aug 04 12:34:21.541505 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.541484    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"]
Aug 04 12:34:21.545393 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.545374    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a7013e-64958cb5dc-r7wpr"]
Aug 04 12:34:21.556871 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.556849    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"]
Aug 04 12:34:21.560949 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:21.560930    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv"]
Aug 04 12:34:22.994092 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:22.994058    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6251a9a3-faec-4cd0-845e-e9ee3eee82b8" path="/var/lib/kubelet/pods/6251a9a3-faec-4cd0-845e-e9ee3eee82b8/volumes"
Aug 04 12:34:22.994524 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:22.994509    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a7a597-d783-456f-a35d-14cedc4c4ab9" path="/var/lib/kubelet/pods/69a7a597-d783-456f-a35d-14cedc4c4ab9/volumes"
Aug 04 12:34:27.388405 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:27.388373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:27.460261 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:27.460225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:27.645526 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:27.645452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:27.910236 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:27.910161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:28.009827 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.009796    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"]
Aug 04 12:34:28.010143 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010132    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a7a597-d783-456f-a35d-14cedc4c4ab9" containerName="maas-api"
Aug 04 12:34:28.010187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010144    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a7a597-d783-456f-a35d-14cedc4c4ab9" containerName="maas-api"
Aug 04 12:34:28.010187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010152    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6251a9a3-faec-4cd0-845e-e9ee3eee82b8" containerName="payload-processing"
Aug 04 12:34:28.010187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010160    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6251a9a3-faec-4cd0-845e-e9ee3eee82b8" containerName="payload-processing"
Aug 04 12:34:28.010187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010170    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a90278e1-74d1-4472-be67-69e5a872569b" containerName="maas-api"
Aug 04 12:34:28.010187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010177    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a90278e1-74d1-4472-be67-69e5a872569b" containerName="maas-api"
Aug 04 12:34:28.010187 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010185    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ae6842-48f2-4700-bf87-b557cff0a55f" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010190    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ae6842-48f2-4700-bf87-b557cff0a55f" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010197    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe2f123-c0eb-4f94-ac85-fdea33aa6e29" containerName="main"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010202    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe2f123-c0eb-4f94-ac85-fdea33aa6e29" containerName="main"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010215    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010220    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010270    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe2f123-c0eb-4f94-ac85-fdea33aa6e29" containerName="main"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010291    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a90278e1-74d1-4472-be67-69e5a872569b" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010296    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a7a597-d783-456f-a35d-14cedc4c4ab9" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010303    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f2b7cf-4e3c-4e23-9f39-3a666aebeae3" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010310    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ae6842-48f2-4700-bf87-b557cff0a55f" containerName="maas-api"
Aug 04 12:34:28.010412 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.010316    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6251a9a3-faec-4cd0-845e-e9ee3eee82b8" containerName="payload-processing"
Aug 04 12:34:28.014665 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.014645    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:28.017502 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.017483    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\""
Aug 04 12:34:28.021774 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.021751    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"]
Aug 04 12:34:28.108139 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.108105    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2abc119-c999-4b0f-85bb-846a66555caa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm\" (UID: \"d2abc119-c999-4b0f-85bb-846a66555caa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:28.209052 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.208939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2abc119-c999-4b0f-85bb-846a66555caa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm\" (UID: \"d2abc119-c999-4b0f-85bb-846a66555caa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:28.209588 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.209571    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2abc119-c999-4b0f-85bb-846a66555caa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm\" (UID: \"d2abc119-c999-4b0f-85bb-846a66555caa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:28.327119 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.327082    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:28.462560 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.462520    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"]
Aug 04 12:34:28.464843 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:34:28.464817    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2abc119_c999_4b0f_85bb_846a66555caa.slice/crio-c594c074a1825bd2058f218f42f032b97e6d1ade758c83956db414c9aecd45b8 WatchSource:0}: Error finding container c594c074a1825bd2058f218f42f032b97e6d1ade758c83956db414c9aecd45b8: Status 404 returned error can't find the container with id c594c074a1825bd2058f218f42f032b97e6d1ade758c83956db414c9aecd45b8
Aug 04 12:34:28.554249 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:28.554216    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm" event={"ID":"d2abc119-c999-4b0f-85bb-846a66555caa","Type":"ContainerStarted","Data":"c594c074a1825bd2058f218f42f032b97e6d1ade758c83956db414c9aecd45b8"}
Aug 04 12:34:29.560536 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:29.560504    2558 generic.go:356] "Generic (PLEG): container finished" podID="d2abc119-c999-4b0f-85bb-846a66555caa" containerID="bc2e5dfe34c641374b9213b3ae6aacc39c80610dd2deb6c7098950a4145201f8" exitCode=0
Aug 04 12:34:29.561015 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:29.560588    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm" event={"ID":"d2abc119-c999-4b0f-85bb-846a66555caa","Type":"ContainerDied","Data":"bc2e5dfe34c641374b9213b3ae6aacc39c80610dd2deb6c7098950a4145201f8"}
Aug 04 12:34:30.698773 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:30.698749    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:30.730992 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:30.730960    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2abc119-c999-4b0f-85bb-846a66555caa-maas-api-service-ca\") pod \"d2abc119-c999-4b0f-85bb-846a66555caa\" (UID: \"d2abc119-c999-4b0f-85bb-846a66555caa\") "
Aug 04 12:34:30.731291 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:30.731258    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2abc119-c999-4b0f-85bb-846a66555caa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2abc119-c999-4b0f-85bb-846a66555caa" (UID: "d2abc119-c999-4b0f-85bb-846a66555caa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:34:30.832597 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:30.832528    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2abc119-c999-4b0f-85bb-846a66555caa-maas-api-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:31.571794 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:31.571757    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm" event={"ID":"d2abc119-c999-4b0f-85bb-846a66555caa","Type":"ContainerDied","Data":"c594c074a1825bd2058f218f42f032b97e6d1ade758c83956db414c9aecd45b8"}
Aug 04 12:34:31.571794 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:31.571787    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"
Aug 04 12:34:31.572016 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:31.571815    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c594c074a1825bd2058f218f42f032b97e6d1ade758c83956db414c9aecd45b8"
Aug 04 12:34:38.438164 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.438119    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"]
Aug 04 12:34:38.438675 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.438413    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" podUID="89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" containerName="maas-api" containerID="cri-o://84399ec6c2cc6ad30c0c2150e2b73eb04052b4bba24643e119770d0b530e3a07" gracePeriod=30
Aug 04 12:34:38.475468 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.475431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:38.505321 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.505267    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"]
Aug 04 12:34:38.505599 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.505545    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft" podUID="8d624b93-da64-41f5-a52b-44a153e7ca4f" containerName="payload-pre-processing" containerID="cri-o://5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0" gracePeriod=30
Aug 04 12:34:38.551299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.551251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:38.607167 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.607137    2558 generic.go:356] "Generic (PLEG): container finished" podID="89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" containerID="84399ec6c2cc6ad30c0c2150e2b73eb04052b4bba24643e119770d0b530e3a07" exitCode=0
Aug 04 12:34:38.607339 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.607168    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" event={"ID":"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b","Type":"ContainerDied","Data":"84399ec6c2cc6ad30c0c2150e2b73eb04052b4bba24643e119770d0b530e3a07"}
Aug 04 12:34:38.650695 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.650657    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"]
Aug 04 12:34:38.653152 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.653122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:38.655682 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.655655    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-038ca6-dkhrm"]
Aug 04 12:34:38.755370 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.755345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:38.759486 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.759466    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:34:38.790332 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.790312    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:34:38.800163 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.800144    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls\") pod \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") "
Aug 04 12:34:38.800242 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.800209    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-openshift-service-ca\") pod \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") "
Aug 04 12:34:38.800311 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.800260    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ks6z\" (UniqueName: \"kubernetes.io/projected/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-kube-api-access-6ks6z\") pod \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\" (UID: \"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b\") "
Aug 04 12:34:38.800590 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.800567    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" (UID: "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:34:38.802260 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.802237    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-kube-api-access-6ks6z" (OuterVolumeSpecName: "kube-api-access-6ks6z") pod "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" (UID: "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b"). InnerVolumeSpecName "kube-api-access-6ks6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:34:38.802348 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.802250    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" (UID: "89cc5de2-fe40-4471-b2b4-a5fb2ff7775b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 12:34:38.901574 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.901543    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d624b93-da64-41f5-a52b-44a153e7ca4f-plugins-config-volume\") pod \"8d624b93-da64-41f5-a52b-44a153e7ca4f\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") "
Aug 04 12:34:38.901744 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.901613    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dlwb\" (UniqueName: \"kubernetes.io/projected/8d624b93-da64-41f5-a52b-44a153e7ca4f-kube-api-access-8dlwb\") pod \"8d624b93-da64-41f5-a52b-44a153e7ca4f\" (UID: \"8d624b93-da64-41f5-a52b-44a153e7ca4f\") "
Aug 04 12:34:38.901796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.901768    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-openshift-service-ca\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:38.901796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.901781    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ks6z\" (UniqueName: \"kubernetes.io/projected/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-kube-api-access-6ks6z\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:38.901796 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.901790    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b-maas-api-tls\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:38.901923 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.901901    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d624b93-da64-41f5-a52b-44a153e7ca4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d624b93-da64-41f5-a52b-44a153e7ca4f" (UID: "8d624b93-da64-41f5-a52b-44a153e7ca4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 12:34:38.903554 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.903533    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d624b93-da64-41f5-a52b-44a153e7ca4f-kube-api-access-8dlwb" (OuterVolumeSpecName: "kube-api-access-8dlwb") pod "8d624b93-da64-41f5-a52b-44a153e7ca4f" (UID: "8d624b93-da64-41f5-a52b-44a153e7ca4f"). InnerVolumeSpecName "kube-api-access-8dlwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 12:34:38.993964 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:38.993883    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2abc119-c999-4b0f-85bb-846a66555caa" path="/var/lib/kubelet/pods/d2abc119-c999-4b0f-85bb-846a66555caa/volumes"
Aug 04 12:34:39.002606 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.002576    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d624b93-da64-41f5-a52b-44a153e7ca4f-plugins-config-volume\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:39.002764 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.002609    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dlwb\" (UniqueName: \"kubernetes.io/projected/8d624b93-da64-41f5-a52b-44a153e7ca4f-kube-api-access-8dlwb\") on node \"ip-10-0-129-11.ec2.internal\" DevicePath \"\""
Aug 04 12:34:39.017826 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.017805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:39.160795 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.160761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:39.613081 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.613049    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"
Aug 04 12:34:39.613876 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.613054    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx" event={"ID":"89cc5de2-fe40-4471-b2b4-a5fb2ff7775b","Type":"ContainerDied","Data":"6a61936b5018d568ee4ad94787edd8bccd7e3c3f30ba0d27fdc0004b703020dc"}
Aug 04 12:34:39.613876 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.613213    2558 scope.go:117] "RemoveContainer" containerID="84399ec6c2cc6ad30c0c2150e2b73eb04052b4bba24643e119770d0b530e3a07"
Aug 04 12:34:39.614403 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.614377    2558 generic.go:356] "Generic (PLEG): container finished" podID="8d624b93-da64-41f5-a52b-44a153e7ca4f" containerID="5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0" exitCode=0
Aug 04 12:34:39.614534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.614416    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"
Aug 04 12:34:39.614534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.614424    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft" event={"ID":"8d624b93-da64-41f5-a52b-44a153e7ca4f","Type":"ContainerDied","Data":"5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0"}
Aug 04 12:34:39.614534 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.614458    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft" event={"ID":"8d624b93-da64-41f5-a52b-44a153e7ca4f","Type":"ContainerDied","Data":"a796f0b337bda06107cf492d87a48de5edc76e5a9914915ab16c6a61c72d4ffc"}
Aug 04 12:34:39.622104 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.622083    2558 scope.go:117] "RemoveContainer" containerID="5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0"
Aug 04 12:34:39.629626 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.629606    2558 scope.go:117] "RemoveContainer" containerID="5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0"
Aug 04 12:34:39.629857 ip-10-0-129-11 kubenswrapper[2558]: E0804 12:34:39.629839    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0\": container with ID starting with 5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0 not found: ID does not exist" containerID="5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0"
Aug 04 12:34:39.629901 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.629865    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0"} err="failed to get container status \"5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0\": rpc error: code = NotFound desc = could not find container \"5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0\": container with ID starting with 5fa98ebff51d9758dd6ee73f47cbb44fdaaa95f72a322812c40bc6152da670a0 not found: ID does not exist"
Aug 04 12:34:39.634246 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.634226    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"]
Aug 04 12:34:39.645966 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.645942    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-038ca6-74bd4579d8-74mhx"]
Aug 04 12:34:39.658373 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.658354    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"]
Aug 04 12:34:39.665455 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:39.665438    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft"]
Aug 04 12:34:40.995482 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:40.995449    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" path="/var/lib/kubelet/pods/89cc5de2-fe40-4471-b2b4-a5fb2ff7775b/volumes"
Aug 04 12:34:40.995902 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:40.995811    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d624b93-da64-41f5-a52b-44a153e7ca4f" path="/var/lib/kubelet/pods/8d624b93-da64-41f5-a52b-44a153e7ca4f/volumes"
Aug 04 12:34:45.772442 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:45.772409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:45.872848 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:45.872819    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:46.074399 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:46.074326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:46.640749 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:46.640709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:34:46.653413 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:46.653393    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84dc45b674-lq5kc_4a39e227-f929-430c-b243-b12973d5576a/authorino/0.log"
Aug 04 12:34:52.157051 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:52.157011    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c67f8cc4c-bh6lm_1ae726b4-46f1-4d8d-887d-97079ffa5dd0/manager/0.log"
Aug 04 12:34:52.755459 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:52.755427    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-vqgwl_6c48cdbf-906c-4a05-a7fb-6c2354dfe639/copy-manifests/0.log"
Aug 04 12:34:52.767062 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:52.767032    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-vqgwl_6c48cdbf-906c-4a05-a7fb-6c2354dfe639/manager/0.log"
Aug 04 12:34:53.094311 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:53.094193    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-78547f99d8-g2tjv_bb828966-ec5d-481c-978c-1f1f9cd39626/maas-api/0.log"
Aug 04 12:34:53.197718 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:53.197690    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764080-jtllf_e2986b88-8d68-49a5-8d31-f32bccb7000d/cleanup/0.log"
Aug 04 12:34:54.292298 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:54.292226    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84dc45b674-lq5kc_4a39e227-f929-430c-b243-b12973d5576a/authorino/0.log"
Aug 04 12:34:54.608408 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:54.608321    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-57tn2_dee418d7-5d82-4a0e-8b0c-f2631cb3207b/kuadrant-console-plugin/0.log"
Aug 04 12:34:55.585726 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:55.585695    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96bfd8458-l6tqk_af47002b-71c8-4f17-9c84-28897ba189aa/kube-auth-proxy/0.log"
Aug 04 12:34:56.047809 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:56.047777    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bf9477994-mzx8r_26e8828f-fa45-4972-8389-98db03ce35f3/router/0.log"
Aug 04 12:34:56.381105 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:56.381009    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nc6km_54f8e3d3-f0bd-4f04-8637-03b9df69d2e3/main/0.log"
Aug 04 12:34:56.806679 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:34:56.806647    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8x4f4_b6350a18-01c0-4d03-ad4a-2465ef52a8d6/main/0.log"
Aug 04 12:35:01.893884 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:01.893851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:35:01.961255 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:01.961193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:35:05.011673 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:05.011635    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j6nsk_7d6006cd-0e2f-4b6b-8771-aa0cbb86f77a/global-pull-secret-syncer/0.log"
Aug 04 12:35:05.108618 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:05.108590    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log"
Aug 04 12:35:05.260152 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:05.260103    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-11.ec2.internal_1ea7ce17e14565481341beac22d82055/haproxy/0.log"
Aug 04 12:35:10.061809 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:10.061775    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84dc45b674-lq5kc_4a39e227-f929-430c-b243-b12973d5576a/authorino/0.log"
Aug 04 12:35:10.169450 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:10.169421    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-57tn2_dee418d7-5d82-4a0e-8b0c-f2631cb3207b/kuadrant-console-plugin/0.log"
Aug 04 12:35:12.322040 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.321959    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-dsb2c_4880a511-b249-4367-9811-55fcbc037524/cluster-monitoring-operator/0.log"
Aug 04 12:35:12.348046 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.348020    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-bkbvv_77cf4aa6-6848-40fa-94b2-4a0907cddfd1/kube-state-metrics/0.log"
Aug 04 12:35:12.377049 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.377016    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-bkbvv_77cf4aa6-6848-40fa-94b2-4a0907cddfd1/kube-rbac-proxy-main/0.log"
Aug 04 12:35:12.399624 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.399598    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-bkbvv_77cf4aa6-6848-40fa-94b2-4a0907cddfd1/kube-rbac-proxy-self/0.log"
Aug 04 12:35:12.648036 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.647958    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9tr8b_da96ae44-9d9e-4f18-84b4-ec7491343169/node-exporter/0.log"
Aug 04 12:35:12.675887 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.675863    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9tr8b_da96ae44-9d9e-4f18-84b4-ec7491343169/kube-rbac-proxy/0.log"
Aug 04 12:35:12.702867 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.702849    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9tr8b_da96ae44-9d9e-4f18-84b4-ec7491343169/init-textfile/0.log"
Aug 04 12:35:12.836583 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.836559    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-kmgl7_2a162196-18d7-4c5b-8bf5-21e87aa23d50/kube-rbac-proxy-main/0.log"
Aug 04 12:35:12.867468 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.867435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-kmgl7_2a162196-18d7-4c5b-8bf5-21e87aa23d50/kube-rbac-proxy-self/0.log"
Aug 04 12:35:12.893258 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:12.893229    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-kmgl7_2a162196-18d7-4c5b-8bf5-21e87aa23d50/openshift-state-metrics/0.log"
Aug 04 12:35:14.739129 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.739099    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-b2z74_a9605480-7983-409a-b5da-e941c4f7b331/networking-console-plugin/0.log"
Aug 04 12:35:14.918635 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.918602    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"]
Aug 04 12:35:14.919082 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919057    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2abc119-c999-4b0f-85bb-846a66555caa" containerName="revoke-keys"
Aug 04 12:35:14.919082 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919078    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2abc119-c999-4b0f-85bb-846a66555caa" containerName="revoke-keys"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919100    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d624b93-da64-41f5-a52b-44a153e7ca4f" containerName="payload-pre-processing"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919110    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d624b93-da64-41f5-a52b-44a153e7ca4f" containerName="payload-pre-processing"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919124    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" containerName="maas-api"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919133    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" containerName="maas-api"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919225    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc5de2-fe40-4471-b2b4-a5fb2ff7775b" containerName="maas-api"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919242    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d624b93-da64-41f5-a52b-44a153e7ca4f" containerName="payload-pre-processing"
Aug 04 12:35:14.919299 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.919252    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2abc119-c999-4b0f-85bb-846a66555caa" containerName="revoke-keys"
Aug 04 12:35:14.924204 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.924185    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:14.927486 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.927467    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8l5dn\"/\"default-dockercfg-b5kh6\""
Aug 04 12:35:14.927661 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.927641    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8l5dn\"/\"openshift-service-ca.crt\""
Aug 04 12:35:14.929785 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.929766    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8l5dn\"/\"kube-root-ca.crt\""
Aug 04 12:35:14.935830 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:14.935808    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"]
Aug 04 12:35:15.013067 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.013039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-sys\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.013221 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.013083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-lib-modules\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.013221 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.013112    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-podres\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.013365 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.013241    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlqjn\" (UniqueName: \"kubernetes.io/projected/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-kube-api-access-qlqjn\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.013365 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.013308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-proc\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114096 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlqjn\" (UniqueName: \"kubernetes.io/projected/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-kube-api-access-qlqjn\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114096 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-proc\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-sys\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-lib-modules\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114220    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-podres\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114259    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-sys\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114392 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114260    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-proc\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114584 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-podres\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.114584 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.114402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-lib-modules\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.122970 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.122934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlqjn\" (UniqueName: \"kubernetes.io/projected/94b2a76e-be07-4bc9-94dd-bf282fa21ba6-kube-api-access-qlqjn\") pod \"perf-node-gather-daemonset-nrg2l\" (UID: \"94b2a76e-be07-4bc9-94dd-bf282fa21ba6\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.234037 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.234003    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.365209 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.365182    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"]
Aug 04 12:35:15.367177 ip-10-0-129-11 kubenswrapper[2558]: W0804 12:35:15.367144    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod94b2a76e_be07_4bc9_94dd_bf282fa21ba6.slice/crio-018134e17d737a5b5a1a73bc9dada10308d0c0306756c3037dfc51038d8ab858 WatchSource:0}: Error finding container 018134e17d737a5b5a1a73bc9dada10308d0c0306756c3037dfc51038d8ab858: Status 404 returned error can't find the container with id 018134e17d737a5b5a1a73bc9dada10308d0c0306756c3037dfc51038d8ab858
Aug 04 12:35:15.410709 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.410681    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/2.log"
Aug 04 12:35:15.415496 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.415476    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-rjvth_5a86e8b9-802f-442f-b6e8-4daf7b39d025/console-operator/3.log"
Aug 04 12:35:15.805991 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.805959    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l" event={"ID":"94b2a76e-be07-4bc9-94dd-bf282fa21ba6","Type":"ContainerStarted","Data":"312a2c89d48b95f98fd928ec7719237e373f64c1b68fe07e38b6431b1c5d9a71"}
Aug 04 12:35:15.806430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.806008    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l" event={"ID":"94b2a76e-be07-4bc9-94dd-bf282fa21ba6","Type":"ContainerStarted","Data":"018134e17d737a5b5a1a73bc9dada10308d0c0306756c3037dfc51038d8ab858"}
Aug 04 12:35:15.806430 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.806058    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:15.822862 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.822814    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l" podStartSLOduration=1.822797365 podStartE2EDuration="1.822797365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:35:15.822097008 +0000 UTC m=+3729.393009815" watchObservedRunningTime="2026-08-04 12:35:15.822797365 +0000 UTC m=+3729.393710169"
Aug 04 12:35:15.977307 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:15.977264    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-rrkd5_bc766a32-2e31-40b2-9ddd-d8294e7235b3/download-server/0.log"
Aug 04 12:35:16.603086 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:16.603058    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-c5vh7_a6126cb4-a498-49b3-bbcd-0d295304093d/volume-data-source-validator/0.log"
Aug 04 12:35:17.764522 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:17.764483    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tgnj9_2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba/dns/0.log"
Aug 04 12:35:17.785156 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:17.785134    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tgnj9_2fa7b61b-6bbd-4e75-9b70-f8aae1aae9ba/kube-rbac-proxy/0.log"
Aug 04 12:35:17.809693 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:17.809668    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8klxl_34457f78-d6ea-4f3b-9061-19bc08898912/dns-node-resolver/0.log"
Aug 04 12:35:18.466831 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:18.466809    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tbxgw_2f3f1bb4-c75e-4b61-9db9-034c63028124/node-ca/0.log"
Aug 04 12:35:19.515915 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:19.515881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:35:19.577791 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:19.577766    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467j9_c69b084c-b802-42fb-bfa4-865368a83757/konnectivity-agent/0.log" limit=1024
Aug 04 12:35:19.638964 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:19.638925    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96bfd8458-l6tqk_af47002b-71c8-4f17-9c84-28897ba189aa/kube-auth-proxy/0.log"
Aug 04 12:35:19.774961 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:19.774870    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bf9477994-mzx8r_26e8828f-fa45-4972-8389-98db03ce35f3/router/0.log"
Aug 04 12:35:20.585428 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:20.585401    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7dznr_15f98335-5a43-4491-839c-11a71132cb4b/serve-healthcheck-canary/0.log"
Aug 04 12:35:21.143456 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:21.143427    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-klg2f_33a40371-2939-448c-a467-d338dad327d1/insights-operator/0.log"
Aug 04 12:35:21.144302 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:21.144275    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-klg2f_33a40371-2939-448c-a467-d338dad327d1/insights-operator/1.log"
Aug 04 12:35:21.165091 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:21.165071    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7k5jv_d2c90932-c01e-4d72-ba89-ebce20441511/kube-rbac-proxy/0.log"
Aug 04 12:35:21.187020 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:21.187001    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7k5jv_d2c90932-c01e-4d72-ba89-ebce20441511/exporter/0.log"
Aug 04 12:35:21.211351 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:21.211328    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7k5jv_d2c90932-c01e-4d72-ba89-ebce20441511/extractor/0.log"
Aug 04 12:35:21.820178 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:21.820149    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-nrg2l"
Aug 04 12:35:24.089937 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:24.089904    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c67f8cc4c-bh6lm_1ae726b4-46f1-4d8d-887d-97079ffa5dd0/manager/0.log"
Aug 04 12:35:24.436211 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:24.436186    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-vqgwl_6c48cdbf-906c-4a05-a7fb-6c2354dfe639/manager/0.log"
Aug 04 12:35:24.456557 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:24.456482    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-vqgwl_6c48cdbf-906c-4a05-a7fb-6c2354dfe639/copy-manifests/0.log"
Aug 04 12:35:25.797449 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:25.797418    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d5b557dbd-dthbm_f8483b2d-b3b8-43c5-9e87-df05963993f7/manager/0.log"
Aug 04 12:35:25.806202 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:25.806179    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96bfd8458-l6tqk_af47002b-71c8-4f17-9c84-28897ba189aa/kube-auth-proxy/0.log"
Aug 04 12:35:25.891292 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:25.891256    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bf9477994-mzx8r_26e8828f-fa45-4972-8389-98db03ce35f3/router/0.log"
Aug 04 12:35:27.582852 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:27.582823    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96bfd8458-l6tqk_af47002b-71c8-4f17-9c84-28897ba189aa/kube-auth-proxy/0.log"
Aug 04 12:35:27.696674 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:27.696643    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bf9477994-mzx8r_26e8828f-fa45-4972-8389-98db03ce35f3/router/0.log"
Aug 04 12:35:31.670715 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:31.670682    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-5jqsp_e45dee60-cff9-46f2-820a-3f7e343524bd/kube-storage-version-migrator-operator/1.log"
Aug 04 12:35:31.672623 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:31.672592    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-5jqsp_e45dee60-cff9-46f2-820a-3f7e343524bd/kube-storage-version-migrator-operator/0.log"
Aug 04 12:35:32.884165 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:32.884139    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c8pjg_9f82d5f7-dfb7-4745-8b34-544761a0f69d/kube-multus-additional-cni-plugins/0.log"
Aug 04 12:35:32.909257 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:32.909231    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c8pjg_9f82d5f7-dfb7-4745-8b34-544761a0f69d/egress-router-binary-copy/0.log"
Aug 04 12:35:32.933141 ip-10-0-129-11 kubenswrapper[2558]: I0804 12:35:32.933119    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c8pjg_9f82d5f7-dfb7-4745-8b34-544761a0f69d/cni-plugins/0.log"