Jul 24 15:45:13.435613 ip-10-0-133-118 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 15:45:13.435628 ip-10-0-133-118 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 15:45:13.435638 ip-10-0-133-118 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 15:45:13.435984 ip-10-0-133-118 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 15:45:23.625779 ip-10-0-133-118 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 15:45:23.625796 ip-10-0-133-118 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1595561ba4e346ef82397767aac6a1b1 --
Jul 24 15:46:55.921661 ip-10-0-133-118 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 15:46:56.389661 ip-10-0-133-118 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:46:56.389661 ip-10-0-133-118 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 15:46:56.389661 ip-10-0-133-118 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:46:56.389661 ip-10-0-133-118 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 15:46:56.389661 ip-10-0-133-118 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:46:56.390825 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.390736    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 15:46:56.395992 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.395972    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:46:56.395992 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.395988    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:46:56.395992 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.395993    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:46:56.395992 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.395997    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396002    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396006    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396009    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396014    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396018    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396022    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396026    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396029    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396034    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396038    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396041    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396057    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396061    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396066    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396072    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396079    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396084    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396088    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396093    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:46:56.396220 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396097    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396102    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396107    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396111    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396116    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396120    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396125    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396129    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396134    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396138    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396144    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396148    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396153    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396158    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396163    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396167    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396172    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396176    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396180    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396184    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:46:56.397101 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396188    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396195    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396199    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396204    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396208    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396212    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396216    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396221    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396225    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396229    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396234    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396238    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396242    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396247    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396250    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396254    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396259    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396263    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396267    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:46:56.397972 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396272    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396291    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396296    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396302    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396307    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396314    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396319    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396323    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396327    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396334    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396339    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396343    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396347    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396352    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396356    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396360    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396367    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396371    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396376    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396381    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:46:56.398797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396386    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396392    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396397    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396402    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396407    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396412    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396416    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396420    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396424    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396431    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396436    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396441    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396446    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396452    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396457    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396461    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.396466    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397702    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397713    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:46:56.399636 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397719    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397724    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397729    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397742    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397747    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397752    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397756    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397760    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397765    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397769    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397773    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397778    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397782    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397787    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397791    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397795    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397799    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397803    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397808    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:46:56.400114 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397812    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397816    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397820    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397827    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397832    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397838    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397842    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397848    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397853    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397858    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397863    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397867    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397871    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397876    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397880    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397885    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397889    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397894    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397898    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:46:56.400610 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397903    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397907    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397911    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397916    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397920    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397924    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397928    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397932    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397936    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397940    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397944    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397949    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397953    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397959    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397965    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397971    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397975    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397979    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397984    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397989    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:46:56.401078 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.397997    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398001    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398006    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398010    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398014    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398019    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398023    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398027    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398031    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398035    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398039    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398046    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398050    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398054    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398059    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398063    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398067    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398071    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398075    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398079    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:46:56.401640 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398083    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398087    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398093    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398099    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398103    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398108    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398112    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398116    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398120    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398124    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398129    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398133    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398139    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398143    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398148    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398152    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398156    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398161    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398165    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398807    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:46:56.402142 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398816    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398821    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398826    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398831    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398836    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398840    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398845    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398849    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398854    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398858    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398862    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398866    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398871    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398875    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398879    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398883    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398888    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398895    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398901    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398906    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:46:56.402759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398910    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398915    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398920    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398925    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398930    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398935    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398940    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398945    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398949    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398954    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398959    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398964    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398969    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398973    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398977    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398982    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398987    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398991    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.398995    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399000    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:46:56.403299 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399004    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399008    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399012    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399018    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399022    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399027    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399031    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399036    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399040    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399044    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399048    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399053    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399057    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399061    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399065    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399069    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399073    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399079    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399086    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:46:56.403798 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399091    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399095    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399099    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399103    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399107    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399111    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399115    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399120    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399123    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399128    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399133    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399137    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399142    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399146    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399151    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399156    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399161    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399164    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399169    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399173    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:46:56.404293 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399177    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399181    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399185    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399189    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399194    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399198    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399202    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399206    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399210    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399214    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399218    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399223    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399227    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399231    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399237    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399242    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399247    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399253    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.399258    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:46:56.404902 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399375    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399386    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399393    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399399    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399405    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399408    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399412    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399417    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399420    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399423    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399427    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399430    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399434    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399437    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399440    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399443    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399445    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399448    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399453    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399456    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399459    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399462    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399466    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399470    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 15:46:56.405401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399476    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399480    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399483    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399486    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399489    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399492    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399495    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399498    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399503    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399506    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399508    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399511    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399514    2577 flags.go:64] FLAG: --enable-server="true"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399518    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399522    2577 flags.go:64] FLAG: --event-burst="100"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399525    2577 flags.go:64] FLAG: --event-qps="50"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399528    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399531    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399534    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399538    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399541    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399544    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399547    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399550    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399553    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 15:46:56.405991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399556    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399559    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399562    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399565    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399567    2577 flags.go:64] FLAG: --feature-gates=""
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399571    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399574    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399579    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399582    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399585    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399588    2577 flags.go:64] FLAG: --help="false"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399591    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399594    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399597    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399600    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399603    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399607    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399610    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399612    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399615    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399618    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399621    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399625    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399627    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 24 15:46:56.406695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399630    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399633    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399636    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399639    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399642    2577 flags.go:64] FLAG: --lock-file=""
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399645    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399648    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399651    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399657    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399659    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399662    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399665    2577 flags.go:64] FLAG: --logging-format="text"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399668    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399671    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399674    2577 flags.go:64] FLAG: --manifest-url=""
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399677    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399682    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399686    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399690    2577 flags.go:64] FLAG: --max-pods="110"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399693    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399696    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399699    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399701    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399704    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399707    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 15:46:56.407327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399710    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399717    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399720    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399725    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399728    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399731    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399737    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399740    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399743    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399746    2577 flags.go:64] FLAG: --port="10250"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399749    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399752    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e3c9a3e1277c9208"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399755    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399758    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399761    2577 flags.go:64] FLAG: --register-node="true"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399764    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399768    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399770    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399773    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399776    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399779    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399782    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399786    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399789    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399792    2577 flags.go:64] FLAG: --runonce="false"
Jul 24 15:46:56.407952 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399794    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399797    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399800    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399803    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399806    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399809    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399813    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399816    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399819    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399821    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399824    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399829    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399831    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399836    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399839    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399844    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399847    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399850    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399854    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399857    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399860    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399862    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399866    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399869    2577 flags.go:64] FLAG: --v="2"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399873    2577 flags.go:64] FLAG: --version="false"
Jul 24 15:46:56.408637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399877    2577 flags.go:64] FLAG: --vmodule=""
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399881    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.399884    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400001    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400005    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400008    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400011    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400014    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400017    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400020    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400023    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400025    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400028    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400031    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400033    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400036    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400039    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400043    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400050    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:46:56.409271 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400053    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400056    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400063    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400066    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400069    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400072    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400074    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400077    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400080    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400082    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400085    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400087    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400091    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400094    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400097    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400099    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400102    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400105    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400107    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:46:56.409759 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400110    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400113    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400115    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400118    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400121    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400124    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400126    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400129    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400131    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400134    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400137    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400139    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400143    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400145    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400148    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400151    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400154    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400158    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400162    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:46:56.410223 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400165    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400167    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400170    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400173    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400175    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400177    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400180    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400182    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400185    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400187    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400190    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400194    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400197    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400200    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400202    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400205    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400208    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400210    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400213    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400215    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:46:56.410712 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400218    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400220    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400223    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400225    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400228    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400231    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400234    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400237    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400240    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400242    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400245    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400248    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400251    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400253    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400256    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400258    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400261    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400264    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400266    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400269    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400271    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:46:56.411211 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400294    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400297    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400301    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.400304    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.401190    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.408788    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.408816    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.408839    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.408848    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408917    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408923    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408928    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408932    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408936    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408940    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:46:56.411797 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408944    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408948    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408951    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408955    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408959    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408963    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408966    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408970    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408973    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408977    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408980    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408984    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408988    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408992    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408995    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.408999    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409003    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409007    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409012    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409015    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:46:56.412178 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409018    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409022    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409026    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409031    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409035    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409038    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409042    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409045    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409049    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409053    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409056    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409060    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409063    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409067    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409071    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409075    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409079    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409083    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409086    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:46:56.412713 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409089    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409095    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409101    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409105    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409109    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409113    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409117    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409120    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409125    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409128    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409132    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409136    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409139    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409144    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409148    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409152    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409156    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409160    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409163    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:46:56.413496 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409168    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409171    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409175    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409179    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409182    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409186    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409190    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409194    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409198    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409201    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409205    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409210    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409214    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409218    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409222    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409226    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409230    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409234    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409237    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409241    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:46:56.414159 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409244    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409248    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409252    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409255    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409259    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409265    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409270    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409287    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409292    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409297    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409302    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409306    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409310    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409314    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:46:56.414698 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409319    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.409326    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409508    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409516    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409520    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409525    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409529    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409533    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409537    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409542    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409546    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409549    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409553    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409556    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409561    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:46:56.415060 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409565    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409569    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409572    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409576    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409580    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409583    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409587    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409591    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409595    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409598    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409602    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409605    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409609    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409614    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409617    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409622    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409626    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409629    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409633    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409637    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:46:56.415475 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409641    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409644    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409647    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409651    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409654    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409658    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409662    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409666    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409669    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409673    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409677    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409681    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409685    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409688    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409692    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409695    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409700    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409704    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409709    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:46:56.415961 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409715    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409719    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409724    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409728    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409732    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409736    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409741    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409745    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409749    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409754    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409758    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409761    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409766    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409770    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409774    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409778    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409781    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409785    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409789    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409793    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:46:56.416481 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409796    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409800    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409803    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409808    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409812    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409816    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409820    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409824    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409827    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409831    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409835    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409838    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409844    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409848    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409852    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409856    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409860    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409864    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409869    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:46:56.416997 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409873    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409877    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409882    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409887    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409891    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409894    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409899    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:56.409902    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.409909    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.410055    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 15:46:56.417476 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.415882    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 15:46:56.417827 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.417495    2577 server.go:1019] "Starting client certificate rotation"
Jul 24 15:46:56.417827 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.417610    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:46:56.417827 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.417651    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:46:56.420218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.420200    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:46:56.421422 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.421372    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:46:56.428042 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.428024    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 24 15:46:56.432136 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.432120    2577 log.go:25] "Validated CRI v1 image API"
Jul 24 15:46:56.435250 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.435234    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 15:46:56.439217 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.439181    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d137e636-fb93-4763-995d-9728718778fc:/dev/nvme0n1p3 fa5c730c-fa2b-4672-8e54-c11fcfa93a8b:/dev/nvme0n1p4]
Jul 24 15:46:56.439331 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.439218    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 15:46:56.447474 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.447434    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:46:56.447702 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.447365    2577 manager.go:216] Machine: {Timestamp:2026-07-24 15:46:56.445449672 +0000 UTC m=+0.381352330 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100693 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e10b2c0abe25790ecf93177b572c SystemUUID:ec27e10b-2c0a-be25-790e-cf93177b572c BootID:1595561b-a4e3-46ef-8239-7767aac6a1b1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:5d:16:66:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:5d:16:66:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:cc:da:db:10:62 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 15:46:56.447702 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.447616    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 15:46:56.447858 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.447839    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 15:46:56.451629 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.451556    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 15:46:56.451898 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.451636    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 15:46:56.452640 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.452627    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 15:46:56.452696 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.452646    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 15:46:56.452696 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.452657    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:46:56.452696 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.452678    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:46:56.454013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.453996    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 15:46:56.456212 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456189    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:46:56.456563 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456547    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 15:46:56.456642 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456612    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 15:46:56.456642 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456631    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 15:46:56.456725 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456663    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 15:46:56.456725 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456686    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 24 15:46:56.456725 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.456705    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 15:46:56.457410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.457392    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:46:56.457504 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.457421    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:46:56.460131 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.460112    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 15:46:56.466131 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.466097    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 15:46:56.466131 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.466136    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 15:46:56.469777 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.469744    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469822    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469837    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469846    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469855    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.469850    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469865    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469890    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469902    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 15:46:56.469908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469908    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 15:46:56.470146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469917    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 15:46:56.470146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469922    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 15:46:56.470146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469934    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 15:46:56.470146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469940    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 15:46:56.470146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469967    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 15:46:56.470146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.469972    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 15:46:56.473604 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.473581    2577 server.go:1268] "Started kubelet"
Jul 24 15:46:56.473779 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.473749    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 15:46:56.473999 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.473726    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 15:46:56.474074 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.474035    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 15:46:56.474636 ip-10-0-133-118 systemd[1]: Started Kubernetes Kubelet.
Jul 24 15:46:56.476187 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.476166    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 15:46:56.476418 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.476399    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 15:46:56.480932 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.480904    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 15:46:56.481090 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.481040    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 15:46:56.482338 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.481169    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-118.ec2.internal.18c5446256a6e763  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-118.ec2.internal,UID:ip-10-0-133-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-118.ec2.internal,},FirstTimestamp:2026-07-24 15:46:56.473532259 +0000 UTC m=+0.409434915,LastTimestamp:2026-07-24 15:46:56.473532259 +0000 UTC m=+0.409434915,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-118.ec2.internal,}"
Jul 24 15:46:56.485734 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.485710    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 15:46:56.485734 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.485727    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 15:46:56.485943 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.485830    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:56.486691 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.486670    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 15:46:56.487864 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.487842    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 15:46:56.487935 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.487894    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 15:46:56.488069 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488052    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 15:46:56.488069 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488060    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 15:46:56.488265 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.488238    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 15:46:56.488552 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488532    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 15:46:56.488552 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488549    2577 factory.go:55] Registering systemd factory
Jul 24 15:46:56.488552 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488556    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 24 15:46:56.488742 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.488676    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 15:46:56.488809 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488792    2577 factory.go:153] Registering CRI-O factory
Jul 24 15:46:56.488809 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488811    2577 factory.go:223] Registration of the crio container factory successfully
Jul 24 15:46:56.488947 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488833    2577 factory.go:103] Registering Raw factory
Jul 24 15:46:56.488947 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.488846    2577 manager.go:1196] Started watching for new ooms in manager
Jul 24 15:46:56.489376 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.489357    2577 manager.go:349] Starting recovery of all containers
Jul 24 15:46:56.493126 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.493091    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gfntd"
Jul 24 15:46:56.497956 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.497918    2577 manager.go:354] Recovery completed
Jul 24 15:46:56.498114 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.498042    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gfntd"
Jul 24 15:46:56.504340 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.504314    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.506331 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506268    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.506331 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506328    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.506437 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506338    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.506897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506877    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 15:46:56.506897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506894    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 15:46:56.507049 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506901    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.507049 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.506913    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:46:56.507635 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.507619    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.507680 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.507650    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.507680 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.507666    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.508672 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.508660    2577 policy_none.go:49] "None policy: Start"
Jul 24 15:46:56.508721 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.508676    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 15:46:56.508721 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.508687    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 15:46:56.509700 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.509690    2577 policy_none.go:47] "Start"
Jul 24 15:46:56.546098 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.545896    2577 manager.go:337] "Starting Device Plugin manager"
Jul 24 15:46:56.546231 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.546126    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 15:46:56.546231 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.546142    2577 server.go:87] "Starting device plugin registration server"
Jul 24 15:46:56.546478 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.546463    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 15:46:56.546539 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.546479    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 15:46:56.546609 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.546580    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 15:46:56.546682 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.546663    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 15:46:56.546682 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.546672    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 15:46:56.547204 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.547179    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 15:46:56.547325 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.547234    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:56.559194 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.559176    2577 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2610 into the system container \"/system.slice\": <nil>"
Jul 24 15:46:56.622117 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.622073    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 15:46:56.623243 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.623231    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 15:46:56.623335 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.623267    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 15:46:56.623335 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.623311    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 15:46:56.623444 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.623350    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 15:46:56.625856 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.625831    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:46:56.646898 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.646814    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.647835 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.647820    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.647911 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.647852    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.647911 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.647861    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.647911 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.647894    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.656586 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.656563    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.656586 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.656591    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-118.ec2.internal\": node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:56.684835 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.684800    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:56.724461 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.724416    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"]
Jul 24 15:46:56.724639 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.724523    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.725695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.725677    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.725796 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.725712    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.725796 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.725725    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.727227 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.727207    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.727343 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.727329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.727393 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.727355    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.728049 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.728033    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.728116 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.728064    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.728116 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.728079    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.728116 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.728086    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.728116 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.728109    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.728299 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.728122    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.729562 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.729542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.729708 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.729581    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:46:56.730462 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.730445    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:46:56.730524 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.730477    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:46:56.730524 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.730487    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:46:56.760181 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.760153    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-118.ec2.internal\" not found" node="ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.764699 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.764682    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-118.ec2.internal\" not found" node="ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.785775 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:56.785742    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:56.789659 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.789640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b20e412873b2e791a1c08e8f485513ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal\" (UID: \"b20e412873b2e791a1c08e8f485513ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.789725 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.789667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7888cd199d1af8b29d9e50b2c19015c9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-118.ec2.internal\" (UID: \"7888cd199d1af8b29d9e50b2c19015c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:56.789725 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:56.789684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b20e412873b2e791a1c08e8f485513ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal\" (UID: \"b20e412873b2e791a1c08e8f485513ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.494857 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:58.494811    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:58.499072 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.499055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7888cd199d1af8b29d9e50b2c19015c9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-118.ec2.internal\" (UID: \"7888cd199d1af8b29d9e50b2c19015c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.499131 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.499079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b20e412873b2e791a1c08e8f485513ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal\" (UID: \"b20e412873b2e791a1c08e8f485513ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.499131 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.499097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b20e412873b2e791a1c08e8f485513ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal\" (UID: \"b20e412873b2e791a1c08e8f485513ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.499131 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.499123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b20e412873b2e791a1c08e8f485513ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal\" (UID: \"b20e412873b2e791a1c08e8f485513ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.499236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.499137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b20e412873b2e791a1c08e8f485513ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal\" (UID: \"b20e412873b2e791a1c08e8f485513ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.499236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.499163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7888cd199d1af8b29d9e50b2c19015c9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-118.ec2.internal\" (UID: \"7888cd199d1af8b29d9e50b2c19015c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.595748 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:58.595661    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:58.671010 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.670986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.676969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.676947    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:58.695799 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:58.695744    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:58.796239 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:58.796197    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:58.896878 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:58.896773    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-118.ec2.internal\" not found"
Jul 24 15:46:58.984876 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.984845    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:46:58.994218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:58.994187    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:59.007318 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.007258    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:46:59.009636 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.009572    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal"
Jul 24 15:46:59.018836 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.018793    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:46:59.024967 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.024932    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 15:46:59.025174 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.025148    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:46:59.067479 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.067445    2577 apiserver.go:52] "Watching apiserver"
Jul 24 15:46:59.077584 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.077548    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 15:46:59.078831 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.078803    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal","openshift-multus/multus-sp4d8","openshift-network-diagnostics/network-check-target-swp2x","openshift-network-operator/iptables-alerter-b8njs","kube-system/global-pull-secret-syncer-lprwc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc","openshift-dns/node-resolver-b2tj6","openshift-multus/multus-additional-cni-plugins-njndp","openshift-multus/network-metrics-daemon-pc2tq","openshift-ovn-kubernetes/ovnkube-node-qgg47","kube-system/konnectivity-agent-49nk2","kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal","openshift-cluster-node-tuning-operator/tuned-svzm9","openshift-image-registry/node-ca-lbzkd"]
Jul 24 15:46:59.080340 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.080314    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.081496 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.081473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.082378 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.082355    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.082492 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.082402    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.082492 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.082452    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w896k\""
Jul 24 15:46:59.082672 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.082652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:46:59.082760 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.082714    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:46:59.083115 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.083100    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.083209 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.083194    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.083580 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.083563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k9m9k\""
Jul 24 15:46:59.083950 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.083935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.085029 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.085012    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.085111 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.085082    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:46:59.085604 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.085583    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.085715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.085595    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.085715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.085689    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tbtg5\""
Jul 24 15:46:59.085827 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.085741    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 15:46:59.086241 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.086225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.087419 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.087400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.087498 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.087459    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:46:59.088766 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.088753    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.089541 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.089522    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.089655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.089543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 15:46:59.089728 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.089672    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5z8kt\""
Jul 24 15:46:59.089728 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.089676    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.090147 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.090135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.090606 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.090591    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lm9gm\""
Jul 24 15:46:59.091123 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.091108    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.091459 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.091446    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.092460 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.092443    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 15:46:59.092703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.092683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.092821 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.092717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bspnx\""
Jul 24 15:46:59.092886 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.092865    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 15:46:59.093091 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093072    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 15:46:59.093165 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093074    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 15:46:59.093165 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093106    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.093165 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093106    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.093428 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093414    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 15:46:59.093896 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.094001 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.093889    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 15:46:59.094169 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094157    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 15:46:59.094236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094193    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.094236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094220    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.094391 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094377    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s9928\""
Jul 24 15:46:59.094569 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094556    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2shtm\""
Jul 24 15:46:59.094647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094639    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 15:46:59.094890 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094876    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 15:46:59.095001 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.094987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 15:46:59.095842 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.095827    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 15:46:59.096104 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.096092    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 15:46:59.096263 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.096248    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-srxxh\""
Jul 24 15:46:59.096360 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.096345    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.096815 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.096795    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 15:46:59.103145 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-socket-dir-parent\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103260 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-cni-bin\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103260 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-etc-kubernetes\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103260 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-cni-bin\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cd59e28-f055-463d-b9be-e61b9928ef7b-host-slash\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-etc-selinux\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20450065-d354-4132-99f9-4f5b0fdc7196-cni-binary-copy\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovnkube-script-lib\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2af47af0-b016-4f57-acc9-918904ff8903-agent-certs\") pod \"konnectivity-agent-49nk2\" (UID: \"2af47af0-b016-4f57-acc9-918904ff8903\") " pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-systemd-units\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103389 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-etc-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-node-log\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-os-release\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-cni-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovnkube-config\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cd59e28-f055-463d-b9be-e61b9928ef7b-iptables-alerter-script\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-os-release\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-k8s-cni-cncf-io\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-cni-multus\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.103647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-slash\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-sys-fs\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-systemd\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-netns\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-system-cni-dir\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-var-lib-kubelet\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpc9d\" (UniqueName: \"kubernetes.io/projected/7eab731c-7e96-45ff-b043-0c3b526cae8e-kube-api-access-hpc9d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20450065-d354-4132-99f9-4f5b0fdc7196-multus-daemon-config\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6r65\" (UniqueName: \"kubernetes.io/projected/97df7b64-4b55-47b8-b30c-956abe38b044-kube-api-access-v6r65\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95304c49-01ee-4b56-8b45-e33b3f2d5b39-kubelet-config\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-ovn\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.103988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-host\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsstx\" (UniqueName: \"kubernetes.io/projected/904f63eb-62b1-407a-99e1-2ade4837bdbb-kube-api-access-qsstx\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.104080 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8xlv\" (UniqueName: \"kubernetes.io/projected/a854e3ef-7cf7-467e-896b-339ed5f3acc6-kube-api-access-g8xlv\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.104642 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.104642 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-conf-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.104642 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-multus-certs\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.104791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-run-netns\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.104791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovn-node-metrics-cert\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.104791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-cni-binary-copy\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.104791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.104995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wkfc\" (UniqueName: \"kubernetes.io/projected/0cd59e28-f055-463d-b9be-e61b9928ef7b-kube-api-access-8wkfc\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.104995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105f600-365c-42f2-bd83-2dc9190a4870-tmp-dir\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.104995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f52x2\" (UniqueName: \"kubernetes.io/projected/8105f600-365c-42f2-bd83-2dc9190a4870-kube-api-access-f52x2\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.104995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.104971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-modprobe-d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.105183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-env-overrides\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.105183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2af47af0-b016-4f57-acc9-918904ff8903-konnectivity-ca\") pod \"konnectivity-agent-49nk2\" (UID: \"2af47af0-b016-4f57-acc9-918904ff8903\") " pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.105183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-cnibin\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.105183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jmv\" (UniqueName: \"kubernetes.io/projected/20450065-d354-4132-99f9-4f5b0fdc7196-kube-api-access-k6jmv\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.105558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-var-lib-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.105558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-kubernetes\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.105558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-systemd\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.105558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-tuned\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.105558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-device-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9043edc6-8df3-47b5-9318-b7a478366ec3-host\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-cnibin\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysctl-d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysctl-conf\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95304c49-01ee-4b56-8b45-e33b3f2d5b39-dbus\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-system-cni-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-kubelet\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-hostroot\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.105873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-kubelet\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-cni-netd\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.105972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysconfig\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-sys\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-log-socket\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-lib-modules\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.106268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-registration-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knf5w\" (UniqueName: \"kubernetes.io/projected/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-kube-api-access-knf5w\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eab731c-7e96-45ff-b043-0c3b526cae8e-tmp\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-run\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-socket-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.106703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106700    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9043edc6-8df3-47b5-9318-b7a478366ec3-serviceca\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.107221 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljw55\" (UniqueName: \"kubernetes.io/projected/9043edc6-8df3-47b5-9318-b7a478366ec3-kube-api-access-ljw55\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.107221 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.106773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8105f600-365c-42f2-bd83-2dc9190a4870-hosts-file\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.108740 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.108717    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:46:59.112477 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.112430    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 15:41:58 +0000 UTC" deadline="2028-01-23 14:52:31.76166496 +0000 UTC"
Jul 24 15:46:59.112477 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.112471    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13151h5m32.649195905s"
Jul 24 15:46:59.140909 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.140880    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pbdqt"
Jul 24 15:46:59.148748 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.148688    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pbdqt"
Jul 24 15:46:59.207736 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jmv\" (UniqueName: \"kubernetes.io/projected/20450065-d354-4132-99f9-4f5b0fdc7196-kube-api-access-k6jmv\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-var-lib-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-kubernetes\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-systemd\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-kubernetes\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-tuned\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-device-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-var-lib-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-systemd\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9043edc6-8df3-47b5-9318-b7a478366ec3-host\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-cnibin\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.207941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207935    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9043edc6-8df3-47b5-9318-b7a478366ec3-host\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysctl-d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.207980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-cnibin\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysctl-conf\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95304c49-01ee-4b56-8b45-e33b3f2d5b39-dbus\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-system-cni-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-kubelet\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-hostroot\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysctl-d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-kubelet\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-system-cni-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-cni-netd\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208162    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysconfig\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-hostroot\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-kubelet\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysctl-conf\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.208532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-sys\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208255    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-cni-netd\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208259    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-sysconfig\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95304c49-01ee-4b56-8b45-e33b3f2d5b39-dbus\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-log-socket\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-kubelet\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-log-socket\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-sys\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-lib-modules\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-registration-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-device-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-registration-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-lib-modules\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knf5w\" (UniqueName: \"kubernetes.io/projected/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-kube-api-access-knf5w\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.209446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eab731c-7e96-45ff-b043-0c3b526cae8e-tmp\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.208581    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.208922    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:59.708881786 +0000 UTC m=+2.036446312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.208986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-run\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-socket-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9043edc6-8df3-47b5-9318-b7a478366ec3-serviceca\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-run\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljw55\" (UniqueName: \"kubernetes.io/projected/9043edc6-8df3-47b5-9318-b7a478366ec3-kube-api-access-ljw55\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8105f600-365c-42f2-bd83-2dc9190a4870-hosts-file\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-socket-dir-parent\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8105f600-365c-42f2-bd83-2dc9190a4870-hosts-file\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-socket-dir\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-cni-bin\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.212411 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-etc-kubernetes\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-cni-bin\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-etc-kubernetes\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-socket-dir-parent\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cd59e28-f055-463d-b9be-e61b9928ef7b-host-slash\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cd59e28-f055-463d-b9be-e61b9928ef7b-host-slash\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-cni-bin\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-etc-selinux\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-cni-bin\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20450065-d354-4132-99f9-4f5b0fdc7196-cni-binary-copy\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-etc-selinux\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovnkube-script-lib\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2af47af0-b016-4f57-acc9-918904ff8903-agent-certs\") pod \"konnectivity-agent-49nk2\" (UID: \"2af47af0-b016-4f57-acc9-918904ff8903\") " pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-systemd-units\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-etc-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-node-log\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-os-release\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-cni-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.213315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovnkube-config\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cd59e28-f055-463d-b9be-e61b9928ef7b-iptables-alerter-script\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209643    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-node-log\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-os-release\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-k8s-cni-cncf-io\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-systemd-units\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-cni-multus\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-var-lib-cni-multus\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-slash\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-sys-fs\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-systemd\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-cni-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-netns\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-system-cni-dir\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-var-lib-kubelet\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.214152 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-systemd\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpc9d\" (UniqueName: \"kubernetes.io/projected/7eab731c-7e96-45ff-b043-0c3b526cae8e-kube-api-access-hpc9d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-etc-openvswitch\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-slash\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovnkube-script-lib\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-os-release\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20450065-d354-4132-99f9-4f5b0fdc7196-multus-daemon-config\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.209981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20450065-d354-4132-99f9-4f5b0fdc7196-cni-binary-copy\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6r65\" (UniqueName: \"kubernetes.io/projected/97df7b64-4b55-47b8-b30c-956abe38b044-kube-api-access-v6r65\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/904f63eb-62b1-407a-99e1-2ade4837bdbb-sys-fs\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-var-lib-kubelet\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95304c49-01ee-4b56-8b45-e33b3f2d5b39-kubelet-config\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-ovn\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210735    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-host\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-netns\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.214808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsstx\" (UniqueName: \"kubernetes.io/projected/904f63eb-62b1-407a-99e1-2ade4837bdbb-kube-api-access-qsstx\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95304c49-01ee-4b56-8b45-e33b3f2d5b39-kubelet-config\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8xlv\" (UniqueName: \"kubernetes.io/projected/a854e3ef-7cf7-467e-896b-339ed5f3acc6-kube-api-access-g8xlv\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20450065-d354-4132-99f9-4f5b0fdc7196-multus-daemon-config\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-conf-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-os-release\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-multus-certs\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-run-netns\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-multus-conf-dir\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovn-node-metrics-cert\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-system-cni-dir\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-cni-binary-copy\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-k8s-cni-cncf-io\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-host\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9043edc6-8df3-47b5-9318-b7a478366ec3-serviceca\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.210925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-run-ovn\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.215284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovnkube-config\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20450065-d354-4132-99f9-4f5b0fdc7196-host-run-multus-certs\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211477    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-host-run-netns\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.211526    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.211575    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:59.711561507 +0000 UTC m=+2.039126026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkfc\" (UniqueName: \"kubernetes.io/projected/0cd59e28-f055-463d-b9be-e61b9928ef7b-kube-api-access-8wkfc\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105f600-365c-42f2-bd83-2dc9190a4870-tmp-dir\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f52x2\" (UniqueName: \"kubernetes.io/projected/8105f600-365c-42f2-bd83-2dc9190a4870-kube-api-access-f52x2\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-modprobe-d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-env-overrides\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2af47af0-b016-4f57-acc9-918904ff8903-konnectivity-ca\") pod \"konnectivity-agent-49nk2\" (UID: \"2af47af0-b016-4f57-acc9-918904ff8903\") " pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-cnibin\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97df7b64-4b55-47b8-b30c-956abe38b044-cni-binary-copy\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.211964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-modprobe-d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-cnibin\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.215808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-env-overrides\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7eab731c-7e96-45ff-b043-0c3b526cae8e-etc-tuned\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eab731c-7e96-45ff-b043-0c3b526cae8e-tmp\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2af47af0-b016-4f57-acc9-918904ff8903-konnectivity-ca\") pod \"konnectivity-agent-49nk2\" (UID: \"2af47af0-b016-4f57-acc9-918904ff8903\") " pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cd59e28-f055-463d-b9be-e61b9928ef7b-iptables-alerter-script\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97df7b64-4b55-47b8-b30c-956abe38b044-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105f600-365c-42f2-bd83-2dc9190a4870-tmp-dir\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.212969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2af47af0-b016-4f57-acc9-918904ff8903-agent-certs\") pod \"konnectivity-agent-49nk2\" (UID: \"2af47af0-b016-4f57-acc9-918904ff8903\") " pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.216348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.214063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-ovn-node-metrics-cert\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.220033 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.219978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jmv\" (UniqueName: \"kubernetes.io/projected/20450065-d354-4132-99f9-4f5b0fdc7196-kube-api-access-k6jmv\") pod \"multus-sp4d8\" (UID: \"20450065-d354-4132-99f9-4f5b0fdc7196\") " pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.223557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljw55\" (UniqueName: \"kubernetes.io/projected/9043edc6-8df3-47b5-9318-b7a478366ec3-kube-api-access-ljw55\") pod \"node-ca-lbzkd\" (UID: \"9043edc6-8df3-47b5-9318-b7a478366ec3\") " pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.223560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f52x2\" (UniqueName: \"kubernetes.io/projected/8105f600-365c-42f2-bd83-2dc9190a4870-kube-api-access-f52x2\") pod \"node-resolver-b2tj6\" (UID: \"8105f600-365c-42f2-bd83-2dc9190a4870\") " pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.223719    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.223738    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.223831    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.223913    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:46:59.723890032 +0000 UTC m=+2.051454567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.223917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knf5w\" (UniqueName: \"kubernetes.io/projected/da1bb5fd-e1fc-4983-842d-4d11b3e0252e-kube-api-access-knf5w\") pod \"ovnkube-node-qgg47\" (UID: \"da1bb5fd-e1fc-4983-842d-4d11b3e0252e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.224072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpc9d\" (UniqueName: \"kubernetes.io/projected/7eab731c-7e96-45ff-b043-0c3b526cae8e-kube-api-access-hpc9d\") pod \"tuned-svzm9\" (UID: \"7eab731c-7e96-45ff-b043-0c3b526cae8e\") " pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.224132 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.224139    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsstx\" (UniqueName: \"kubernetes.io/projected/904f63eb-62b1-407a-99e1-2ade4837bdbb-kube-api-access-qsstx\") pod \"aws-ebs-csi-driver-node-6gxmc\" (UID: \"904f63eb-62b1-407a-99e1-2ade4837bdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.225432 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.225321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6r65\" (UniqueName: \"kubernetes.io/projected/97df7b64-4b55-47b8-b30c-956abe38b044-kube-api-access-v6r65\") pod \"multus-additional-cni-plugins-njndp\" (UID: \"97df7b64-4b55-47b8-b30c-956abe38b044\") " pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.225542 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.225437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8xlv\" (UniqueName: \"kubernetes.io/projected/a854e3ef-7cf7-467e-896b-339ed5f3acc6-kube-api-access-g8xlv\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.229897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.229728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wkfc\" (UniqueName: \"kubernetes.io/projected/0cd59e28-f055-463d-b9be-e61b9928ef7b-kube-api-access-8wkfc\") pod \"iptables-alerter-b8njs\" (UID: \"0cd59e28-f055-463d-b9be-e61b9928ef7b\") " pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.240894 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.240873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-njndp"
Jul 24 15:46:59.316190 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.316145    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7888cd199d1af8b29d9e50b2c19015c9.slice/crio-a625c1ac0552e86b18043abf834218413cb43abd91bfbbb1bc2e2470094eb267 WatchSource:0}: Error finding container a625c1ac0552e86b18043abf834218413cb43abd91bfbbb1bc2e2470094eb267: Status 404 returned error can't find the container with id a625c1ac0552e86b18043abf834218413cb43abd91bfbbb1bc2e2470094eb267
Jul 24 15:46:59.316433 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.316401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20e412873b2e791a1c08e8f485513ff.slice/crio-5aa698a3e6057e2da408f4473ab17a50058b8e5ea6673f82a7774bb9d88e1ad2 WatchSource:0}: Error finding container 5aa698a3e6057e2da408f4473ab17a50058b8e5ea6673f82a7774bb9d88e1ad2: Status 404 returned error can't find the container with id 5aa698a3e6057e2da408f4473ab17a50058b8e5ea6673f82a7774bb9d88e1ad2
Jul 24 15:46:59.321774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.321751    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:46:59.336923 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.336890    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:46:59.409374 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.409208    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b2tj6"
Jul 24 15:46:59.421042 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.420969    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8105f600_365c_42f2_bd83_2dc9190a4870.slice/crio-b8d7dbd1a2df3b0fb48d74c7efa2e55c5e40bdb4bde09428f2336c43f217b77d WatchSource:0}: Error finding container b8d7dbd1a2df3b0fb48d74c7efa2e55c5e40bdb4bde09428f2336c43f217b77d: Status 404 returned error can't find the container with id b8d7dbd1a2df3b0fb48d74c7efa2e55c5e40bdb4bde09428f2336c43f217b77d
Jul 24 15:46:59.424563 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.424524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-svzm9"
Jul 24 15:46:59.432592 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.432546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b8njs"
Jul 24 15:46:59.434318 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.434287    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eab731c_7e96_45ff_b043_0c3b526cae8e.slice/crio-cd03452574f075ee58a0e5a336e5c65690d5c93a46f4c15d441dceefe41ffb16 WatchSource:0}: Error finding container cd03452574f075ee58a0e5a336e5c65690d5c93a46f4c15d441dceefe41ffb16: Status 404 returned error can't find the container with id cd03452574f075ee58a0e5a336e5c65690d5c93a46f4c15d441dceefe41ffb16
Jul 24 15:46:59.441753 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.441709    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd59e28_f055_463d_b9be_e61b9928ef7b.slice/crio-eb5eda8b26a3d78548cb66bdccc6d91612330cc0fc03596608f86b9865ed9554 WatchSource:0}: Error finding container eb5eda8b26a3d78548cb66bdccc6d91612330cc0fc03596608f86b9865ed9554: Status 404 returned error can't find the container with id eb5eda8b26a3d78548cb66bdccc6d91612330cc0fc03596608f86b9865ed9554
Jul 24 15:46:59.444685 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.444659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc"
Jul 24 15:46:59.452968 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.452935    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904f63eb_62b1_407a_99e1_2ade4837bdbb.slice/crio-63424895093df90b50049b8052a0397afb4f4ffb13792def769c11718cc2199c WatchSource:0}: Error finding container 63424895093df90b50049b8052a0397afb4f4ffb13792def769c11718cc2199c: Status 404 returned error can't find the container with id 63424895093df90b50049b8052a0397afb4f4ffb13792def769c11718cc2199c
Jul 24 15:46:59.465845 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.465804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lbzkd"
Jul 24 15:46:59.474254 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.474220    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9043edc6_8df3_47b5_9318_b7a478366ec3.slice/crio-e941791c7e687f82598afa82e16c9f61507cc09f74b3bd4b1218ae24cab76fbc WatchSource:0}: Error finding container e941791c7e687f82598afa82e16c9f61507cc09f74b3bd4b1218ae24cab76fbc: Status 404 returned error can't find the container with id e941791c7e687f82598afa82e16c9f61507cc09f74b3bd4b1218ae24cab76fbc
Jul 24 15:46:59.484186 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.484162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sp4d8"
Jul 24 15:46:59.491927 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.491889    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20450065_d354_4132_99f9_4f5b0fdc7196.slice/crio-16b9baf29e3e1644cc67840af603584c14d5393d67b01bb3d586af96e9fc2e88 WatchSource:0}: Error finding container 16b9baf29e3e1644cc67840af603584c14d5393d67b01bb3d586af96e9fc2e88: Status 404 returned error can't find the container with id 16b9baf29e3e1644cc67840af603584c14d5393d67b01bb3d586af96e9fc2e88
Jul 24 15:46:59.503542 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.503517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:46:59.508436 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.508417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:46:59.510914 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.510885    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1bb5fd_e1fc_4983_842d_4d11b3e0252e.slice/crio-dfbccc5912d74492e0d3a9534e715bc003696885c1966e6330b3a4e558b10027 WatchSource:0}: Error finding container dfbccc5912d74492e0d3a9534e715bc003696885c1966e6330b3a4e558b10027: Status 404 returned error can't find the container with id dfbccc5912d74492e0d3a9534e715bc003696885c1966e6330b3a4e558b10027
Jul 24 15:46:59.515362 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.515339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af47af0_b016_4f57_acc9_918904ff8903.slice/crio-bcb6329f22528700a73aa4f2a7a5d476c9b86a640ec455650633173777f53c3e WatchSource:0}: Error finding container bcb6329f22528700a73aa4f2a7a5d476c9b86a640ec455650633173777f53c3e: Status 404 returned error can't find the container with id bcb6329f22528700a73aa4f2a7a5d476c9b86a640ec455650633173777f53c3e
Jul 24 15:46:59.562138 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:46:59.562092    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97df7b64_4b55_47b8_b30c_956abe38b044.slice/crio-abee22f39af2f09b3d137c78d1633d8c07da0902e1ef14cbecd4d436bba0c324 WatchSource:0}: Error finding container abee22f39af2f09b3d137c78d1633d8c07da0902e1ef14cbecd4d436bba0c324: Status 404 returned error can't find the container with id abee22f39af2f09b3d137c78d1633d8c07da0902e1ef14cbecd4d436bba0c324
Jul 24 15:46:59.579346 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.579301    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:46:59.716160 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.715974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:46:59.716160 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.716053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:46:59.716487 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.716189    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:46:59.716487 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.716189    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:46:59.716487 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.716274    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:00.716252025 +0000 UTC m=+3.043816542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:46:59.716487 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.716343    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:00.716327193 +0000 UTC m=+3.043891719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:46:59.816595 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.816554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:46:59.816789 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.816730    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:46:59.816789 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.816751    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:46:59.816789 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.816766    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:46:59.816985 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:46:59.816840    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:47:00.816819955 +0000 UTC m=+3.144384489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:46:59.927084 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:46:59.926957    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:47:00.151816 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.151537    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:41:59 +0000 UTC" deadline="2028-02-07 06:00:57.051758709 +0000 UTC"
Jul 24 15:47:00.151816 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.151646    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13502h13m56.900120384s"
Jul 24 15:47:00.234475 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.233258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:00.234475 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.233406    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:00.234475 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.233892    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:00.234475 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.234070    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:00.269981 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.269797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerStarted","Data":"abee22f39af2f09b3d137c78d1633d8c07da0902e1ef14cbecd4d436bba0c324"}
Jul 24 15:47:00.281348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.281180    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"dfbccc5912d74492e0d3a9534e715bc003696885c1966e6330b3a4e558b10027"}
Jul 24 15:47:00.306497 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.306454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sp4d8" event={"ID":"20450065-d354-4132-99f9-4f5b0fdc7196","Type":"ContainerStarted","Data":"16b9baf29e3e1644cc67840af603584c14d5393d67b01bb3d586af96e9fc2e88"}
Jul 24 15:47:00.309311 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.309246    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lbzkd" event={"ID":"9043edc6-8df3-47b5-9318-b7a478366ec3","Type":"ContainerStarted","Data":"e941791c7e687f82598afa82e16c9f61507cc09f74b3bd4b1218ae24cab76fbc"}
Jul 24 15:47:00.321723 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.321592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc" event={"ID":"904f63eb-62b1-407a-99e1-2ade4837bdbb","Type":"ContainerStarted","Data":"63424895093df90b50049b8052a0397afb4f4ffb13792def769c11718cc2199c"}
Jul 24 15:47:00.328901 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.328829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b8njs" event={"ID":"0cd59e28-f055-463d-b9be-e61b9928ef7b","Type":"ContainerStarted","Data":"eb5eda8b26a3d78548cb66bdccc6d91612330cc0fc03596608f86b9865ed9554"}
Jul 24 15:47:00.348188 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.348147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-svzm9" event={"ID":"7eab731c-7e96-45ff-b043-0c3b526cae8e","Type":"ContainerStarted","Data":"cd03452574f075ee58a0e5a336e5c65690d5c93a46f4c15d441dceefe41ffb16"}
Jul 24 15:47:00.358525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.358172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal" event={"ID":"b20e412873b2e791a1c08e8f485513ff","Type":"ContainerStarted","Data":"5aa698a3e6057e2da408f4473ab17a50058b8e5ea6673f82a7774bb9d88e1ad2"}
Jul 24 15:47:00.374363 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.374233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49nk2" event={"ID":"2af47af0-b016-4f57-acc9-918904ff8903","Type":"ContainerStarted","Data":"bcb6329f22528700a73aa4f2a7a5d476c9b86a640ec455650633173777f53c3e"}
Jul 24 15:47:00.387683 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.387586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b2tj6" event={"ID":"8105f600-365c-42f2-bd83-2dc9190a4870","Type":"ContainerStarted","Data":"b8d7dbd1a2df3b0fb48d74c7efa2e55c5e40bdb4bde09428f2336c43f217b77d"}
Jul 24 15:47:00.394333 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.394267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal" event={"ID":"7888cd199d1af8b29d9e50b2c19015c9","Type":"ContainerStarted","Data":"a625c1ac0552e86b18043abf834218413cb43abd91bfbbb1bc2e2470094eb267"}
Jul 24 15:47:00.724951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.724913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:00.725146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.724991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:00.725146 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.725113    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:00.725247 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.725175    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:02.725156221 +0000 UTC m=+5.052720737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:00.725643 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.725607    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:00.725757 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.725674    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:02.725658874 +0000 UTC m=+5.053223390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:00.825949 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:00.825905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:00.826172 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.826113    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:47:00.826172 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.826135    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:47:00.826172 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.826151    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:00.826336 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:00.826212    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:47:02.826193574 +0000 UTC m=+5.153758113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:01.152468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:01.152370    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:41:59 +0000 UTC" deadline="2028-04-14 22:58:30.462179068 +0000 UTC"
Jul 24 15:47:01.152468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:01.152418    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15127h11m29.309767168s"
Jul 24 15:47:01.232783 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:01.232750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:01.233001 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:01.232900    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:02.235386 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:02.234739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:02.235386 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.234925    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:02.235386 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:02.234979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:02.235386 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.235131    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:02.743120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:02.743078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:02.743292 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:02.743146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:02.743292 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.743262    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:02.743418 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.743321    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:06.743304731 +0000 UTC m=+9.070869247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:02.743722 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.743702    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:02.743813 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.743754    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:06.743740989 +0000 UTC m=+9.071305507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:02.844052 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:02.843967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:02.844237 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.844181    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:47:02.844237 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.844199    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:47:02.844237 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.844213    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:02.844437 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:02.844272    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:47:06.844254164 +0000 UTC m=+9.171818700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:03.232516 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:03.232272    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:03.232516 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:03.232409    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:04.232699 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:04.232588    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:04.232699 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:04.232632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:04.233268 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:04.232765    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:04.236964 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:04.234155    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:05.232298 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:05.232258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:05.232499 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:05.232402    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:06.235658 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:06.235608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:06.235658 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:06.235654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:06.236070 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.235761    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:06.236183 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.236151    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:06.776820 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:06.776761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:06.776991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:06.776856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:06.776991 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.776914    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:06.776991 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.776987    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:06.777127 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.776989    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:14.776967652 +0000 UTC m=+17.104532182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:06.777127 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.777050    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:14.777032152 +0000 UTC m=+17.104596672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:06.878229 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:06.878194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:06.878424 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.878380    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:47:06.878424 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.878403    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:47:06.878424 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.878416    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:06.878573 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:06.878483    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:47:14.87846589 +0000 UTC m=+17.206030420 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:07.232459 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:07.232425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:07.232663 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:07.232552    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:08.236294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:08.235928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:08.236294 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:08.236059    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:08.236294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:08.236079    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:08.236294 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:08.236210    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:09.232718 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:09.232608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:09.232889 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:09.232760    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:10.232940 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:10.232898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:10.233463 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:10.233049    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:10.233463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:10.233082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:10.233463 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:10.233212    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:11.232443 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:11.232416    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:11.232590 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:11.232512    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:12.232489 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:12.232457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:12.232971 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:12.232600    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:12.232971 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:12.232650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:12.232971 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:12.232768    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:13.232136 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:13.232094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:13.232323 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:13.232243    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:14.232831 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:14.232789    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:14.233267 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.232922    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:14.233267 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:14.232956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:14.233267 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.233018    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:14.841256 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:14.841217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:14.841599 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:14.841297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:14.841599 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.841441    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:14.841599 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.841442    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:14.841599 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.841539    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:30.841512726 +0000 UTC m=+33.169077244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:14.841836 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.841609    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:30.841590506 +0000 UTC m=+33.169155023 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:14.942414 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:14.942377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:14.942575 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.942524    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:47:14.942575 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.942538    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:47:14.942575 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.942547    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:14.942696 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:14.942601    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:47:30.94258855 +0000 UTC m=+33.270153067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:15.232738 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:15.232658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:15.232901 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:15.232766    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:16.232629 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:16.232584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:16.232899 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:16.232591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:16.232899 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:16.232742    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:16.232899 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:16.232779    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:17.231978 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:17.231929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:17.232178 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:17.232062    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:18.232860 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:18.232819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:18.233320 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:18.232920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:18.233320 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:18.232932    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:18.233320 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:18.233022    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:19.232178 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:19.232112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:19.232557 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:19.232288    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:20.232384 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.232087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:20.233106 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.232112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:20.233106 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:20.232473    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:20.233106 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:20.232554    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:20.461338 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.461218    2577 generic.go:356] "Generic (PLEG): container finished" podID="97df7b64-4b55-47b8-b30c-956abe38b044" containerID="d349fa06d4d8b943607c86e72a889a7308ce3509b159300d66a631530f30d9ba" exitCode=0
Jul 24 15:47:20.461338 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.461317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerDied","Data":"d349fa06d4d8b943607c86e72a889a7308ce3509b159300d66a631530f30d9ba"}
Jul 24 15:47:20.464897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.464862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"a65824f7500fcad6d842e2713aa6678dec45c9e471f33b3aa1135c386c782c5a"}
Jul 24 15:47:20.464999 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.464936    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"ad6b1ba800bb7ae0c952bbfe78302150ca697abe706842ff645a584a3834fc3f"}
Jul 24 15:47:20.465055 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.465001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"ad0706ddc4c024b81c6fe1d3a4d1704cc210766b622217ff94ff26df07c44565"}
Jul 24 15:47:20.465055 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.465040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"a2049e59e9eeb32adef183c5c74faa02df6551c5fdefb9c4c1fa66bea622a984"}
Jul 24 15:47:20.465358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.465070    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"8c8bbc41dfba5d598dfbadc25380fb174df33ddea4bf873b67df50040790f35b"}
Jul 24 15:47:20.465358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.465097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"41cef074efca1036e52e4b2e7a8291bae911780e16131219186983f352869577"}
Jul 24 15:47:20.469386 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.469353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sp4d8" event={"ID":"20450065-d354-4132-99f9-4f5b0fdc7196","Type":"ContainerStarted","Data":"835bc28f9bfbc48f065e80711f0da9179367ebc98a7d2e1e0e2b3fc62cfdcdb8"}
Jul 24 15:47:20.471059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.471014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lbzkd" event={"ID":"9043edc6-8df3-47b5-9318-b7a478366ec3","Type":"ContainerStarted","Data":"664c1b1d4f44aa9cd2bc7cab58a74c75caf30c591cade9980338d49a50ac35a5"}
Jul 24 15:47:20.472525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.472496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc" event={"ID":"904f63eb-62b1-407a-99e1-2ade4837bdbb","Type":"ContainerStarted","Data":"5493b08f1d4c0b18e7feeb1535a6d8892b0459d326fbf90c430791fe983b4a13"}
Jul 24 15:47:20.473832 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.473807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-svzm9" event={"ID":"7eab731c-7e96-45ff-b043-0c3b526cae8e","Type":"ContainerStarted","Data":"f7823554cc9ede59e8e70dd50257b3739871b97984c3b0c2d5b5b940f7873cfb"}
Jul 24 15:47:20.475364 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.475340    2577 generic.go:356] "Generic (PLEG): container finished" podID="b20e412873b2e791a1c08e8f485513ff" containerID="8f698bc9eae2e48b9a752f4152e9c163d2d7dc61f7e9f2869574b4f4423243e3" exitCode=0
Jul 24 15:47:20.475481 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.475408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal" event={"ID":"b20e412873b2e791a1c08e8f485513ff","Type":"ContainerDied","Data":"8f698bc9eae2e48b9a752f4152e9c163d2d7dc61f7e9f2869574b4f4423243e3"}
Jul 24 15:47:20.477222 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.477177    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49nk2" event={"ID":"2af47af0-b016-4f57-acc9-918904ff8903","Type":"ContainerStarted","Data":"3ba36db5dcf4df72850dd85b23014de6adae939e1217997122fd280feafbef11"}
Jul 24 15:47:20.478677 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.478653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b2tj6" event={"ID":"8105f600-365c-42f2-bd83-2dc9190a4870","Type":"ContainerStarted","Data":"33df49f6adc7510d2779f6c7e95a53e250db74570b41c2672f60fddaffe4296f"}
Jul 24 15:47:20.480236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.480215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal" event={"ID":"7888cd199d1af8b29d9e50b2c19015c9","Type":"ContainerStarted","Data":"da6864c2e3ce196ceea049117fa16ac65846a8adbe4cc18e9c62b0e1390379d3"}
Jul 24 15:47:20.506272 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.506114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-svzm9" podStartSLOduration=2.672028617 podStartE2EDuration="22.5060959s" totalImagesPullingTime="19.834067283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:20.505973996 +0000 UTC m=+22.833538534" watchObservedRunningTime="2026-07-24 15:47:20.5060959 +0000 UTC m=+22.833660438"
Jul 24 15:47:20.506272 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.506219    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lbzkd" podStartSLOduration=2.741033417 podStartE2EDuration="22.506212087s" totalImagesPullingTime="19.76517867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:20.492147764 +0000 UTC m=+22.819712302" watchObservedRunningTime="2026-07-24 15:47:20.506212087 +0000 UTC m=+22.833776662"
Jul 24 15:47:20.525078 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.525017    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sp4d8" podStartSLOduration=2.745415549 podStartE2EDuration="22.524987895s" totalImagesPullingTime="19.779572346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:20.524848812 +0000 UTC m=+22.852413374" watchObservedRunningTime="2026-07-24 15:47:20.524987895 +0000 UTC m=+22.852552434"
Jul 24 15:47:20.550899 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.550847    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-49nk2" podStartSLOduration=3.00806561 podStartE2EDuration="22.550828724s" totalImagesPullingTime="19.542763114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:20.537580318 +0000 UTC m=+22.865144883" watchObservedRunningTime="2026-07-24 15:47:20.550828724 +0000 UTC m=+22.878393262"
Jul 24 15:47:20.578712 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.578590    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b2tj6" podStartSLOduration=2.76023513 podStartE2EDuration="22.578558964s" totalImagesPullingTime="19.818323834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:20.562398988 +0000 UTC m=+22.889963525" watchObservedRunningTime="2026-07-24 15:47:20.578558964 +0000 UTC m=+22.906123504"
Jul 24 15:47:20.579149 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:20.579120    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-118.ec2.internal" podStartSLOduration=21.57910539 podStartE2EDuration="21.57910539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:20.577967423 +0000 UTC m=+22.905531971" watchObservedRunningTime="2026-07-24 15:47:20.57910539 +0000 UTC m=+22.906669927"
Jul 24 15:47:21.071674 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.071580    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:47:21.072362 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.072344    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:47:21.232931 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.232901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:21.233368 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:21.233009    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:21.462978 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.462933    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 15:47:21.484985 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.484903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc" event={"ID":"904f63eb-62b1-407a-99e1-2ade4837bdbb","Type":"ContainerStarted","Data":"60289dab330d5035544a823354b25341fb821b55f2b9cea46286c69edd11d392"}
Jul 24 15:47:21.486837 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.486769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b8njs" event={"ID":"0cd59e28-f055-463d-b9be-e61b9928ef7b","Type":"ContainerStarted","Data":"f2b764988c4e30c595f0602883abb5acc9b84cf7699e50150a9aa06c9f1c6a9b"}
Jul 24 15:47:21.490187 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.490146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal" event={"ID":"b20e412873b2e791a1c08e8f485513ff","Type":"ContainerStarted","Data":"2588f0d78a9a92c14ebbb310ede233f9b62456fe60e4c760d714cebb9ddce6e9"}
Jul 24 15:47:21.501589 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.501538    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b8njs" podStartSLOduration=3.885426028 podStartE2EDuration="23.501518992s" totalImagesPullingTime="19.616092964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:21.500944657 +0000 UTC m=+23.828509203" watchObservedRunningTime="2026-07-24 15:47:21.501518992 +0000 UTC m=+23.829083530"
Jul 24 15:47:21.513396 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:21.513343    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-118.ec2.internal" podStartSLOduration=22.513321481 podStartE2EDuration="22.513321481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:21.513138913 +0000 UTC m=+23.840703452" watchObservedRunningTime="2026-07-24 15:47:21.513321481 +0000 UTC m=+23.840886018"
Jul 24 15:47:22.167780 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.167678    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T15:47:21.462966444Z","UUID":"9fed18ed-affb-4f8a-835b-8f9cd5969270","Handler":null,"Name":"","Endpoint":""}
Jul 24 15:47:22.169284 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.169258    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 15:47:22.169471 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.169294    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 15:47:22.233124 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.233079    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:22.233864 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.233075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:22.233864 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:22.233275    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:22.233864 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:22.233314    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:22.499071 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.499043    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 15:47:22.499538 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:22.499513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"4eab366e3419fa098da28e5698e693211e67f54804407bc22a60cc921f863810"}
Jul 24 15:47:23.232394 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:23.232366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:23.232527 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:23.232490    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:23.504685 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:23.504575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc" event={"ID":"904f63eb-62b1-407a-99e1-2ade4837bdbb","Type":"ContainerStarted","Data":"7c397143bf9571847bc04e3c87f585d0d5b786a6fcd3cb5adc0ef93c75b5e773"}
Jul 24 15:47:23.521873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:23.521796    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6gxmc" podStartSLOduration=2.935588713 podStartE2EDuration="25.521779214s" totalImagesPullingTime="22.586190501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:23.521198583 +0000 UTC m=+25.848763121" watchObservedRunningTime="2026-07-24 15:47:23.521779214 +0000 UTC m=+25.849343751"
Jul 24 15:47:23.834475 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:23.834381    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:47:23.834636 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:23.834520    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 15:47:23.835222 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:23.835024    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-49nk2"
Jul 24 15:47:24.232032 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:24.231948    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:24.232201 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:24.231955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:24.232201 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:24.232097    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:24.232201 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:24.232167    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:25.232234 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.231961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:25.233181 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:25.232319    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:25.511026 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.510989    2577 generic.go:356] "Generic (PLEG): container finished" podID="97df7b64-4b55-47b8-b30c-956abe38b044" containerID="1c462626de17b7686004eda71f6519fcc7ae64b265a5ed74fb0bbdb58709dafb" exitCode=0
Jul 24 15:47:25.511202 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.511079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerDied","Data":"1c462626de17b7686004eda71f6519fcc7ae64b265a5ed74fb0bbdb58709dafb"}
Jul 24 15:47:25.515221 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.515192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" event={"ID":"da1bb5fd-e1fc-4983-842d-4d11b3e0252e","Type":"ContainerStarted","Data":"7b291699bd96db2b94edc749ab27959c15c8b09b0b2afdbdcccd095bb24186d1"}
Jul 24 15:47:25.515485 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.515461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:47:25.531823 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.531796    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:47:25.579027 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:25.578974    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47" podStartSLOduration=7.541665737 podStartE2EDuration="27.578958084s" totalImagesPullingTime="20.037292347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:25.556300415 +0000 UTC m=+27.883864973" watchObservedRunningTime="2026-07-24 15:47:25.578958084 +0000 UTC m=+27.906522627"
Jul 24 15:47:26.232047 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:26.232004    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:26.232203 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:26.232052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:26.232203 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:26.232168    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:26.232786 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:26.232292    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:26.518703 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:26.518392    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:47:26.519115 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:26.519093    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:47:26.535236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:26.535204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:47:27.232763 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.232690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:27.233110 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:27.232793    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:27.521889 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.521811    2577 generic.go:356] "Generic (PLEG): container finished" podID="97df7b64-4b55-47b8-b30c-956abe38b044" containerID="90096e2ca87aa4cf5661d00c18e9869734fd56c67a5f78da91e414670688808e" exitCode=0
Jul 24 15:47:27.522033 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.521903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerDied","Data":"90096e2ca87aa4cf5661d00c18e9869734fd56c67a5f78da91e414670688808e"}
Jul 24 15:47:27.587818 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.587785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lprwc"]
Jul 24 15:47:27.587995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.587917    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:27.588054 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:27.588030    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:27.591367 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.591341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pc2tq"]
Jul 24 15:47:27.591495 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.591469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:27.591603 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:27.591584    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:27.592042 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.592021    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-swp2x"]
Jul 24 15:47:27.592122 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:27.592100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:27.592211 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:27.592190    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:29.232594 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:29.232555    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:29.233019 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:29.232567    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:29.233019 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:29.232577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:29.233019 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:29.232712    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:29.233019 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:29.232769    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:29.233019 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:29.232851    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:29.529225 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:29.529133    2577 generic.go:356] "Generic (PLEG): container finished" podID="97df7b64-4b55-47b8-b30c-956abe38b044" containerID="4962b1dd860208269b159db201d2a8b68ffa1b08526c654c3c81cb66ca7df430" exitCode=0
Jul 24 15:47:29.529225 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:29.529179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerDied","Data":"4962b1dd860208269b159db201d2a8b68ffa1b08526c654c3c81cb66ca7df430"}
Jul 24 15:47:30.869541 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:30.869449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:30.869541 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:30.869530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:30.869989 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.869598    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:30.869989 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.869654    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:30.869989 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.869687    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:02.869664023 +0000 UTC m=+65.197228561 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:47:30.869989 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.869707    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret podName:95304c49-01ee-4b56-8b45-e33b3f2d5b39 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:02.869698171 +0000 UTC m=+65.197262693 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret") pod "global-pull-secret-syncer-lprwc" (UID: "95304c49-01ee-4b56-8b45-e33b3f2d5b39") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:47:30.969914 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:30.969876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:30.970094 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.970010    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:47:30.970094 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.970033    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:47:30.970094 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.970046    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qpp76 for pod openshift-network-diagnostics/network-check-target-swp2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:30.970270 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:30.970107    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76 podName:a2e14ee3-4a59-4833-bab8-73545f677c9c nodeName:}" failed. No retries permitted until 2026-07-24 15:48:02.97009091 +0000 UTC m=+65.297655441 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpp76" (UniqueName: "kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76") pod "network-check-target-swp2x" (UID: "a2e14ee3-4a59-4833-bab8-73545f677c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:47:31.232117 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:31.232032    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:31.232302 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:31.232157    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lprwc" podUID="95304c49-01ee-4b56-8b45-e33b3f2d5b39"
Jul 24 15:47:31.232302 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:31.232032    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:31.232302 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:31.232256    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-swp2x" podUID="a2e14ee3-4a59-4833-bab8-73545f677c9c"
Jul 24 15:47:31.232302 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:31.232032    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:31.232703 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:31.232681    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pc2tq" podUID="a854e3ef-7cf7-467e-896b-339ed5f3acc6"
Jul 24 15:47:32.495732 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.495489    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-118.ec2.internal" event="NodeReady"
Jul 24 15:47:32.496107 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.495787    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 15:47:32.528982 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.528939    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68dbdf489c-5rvb6"]
Jul 24 15:47:32.533590 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.533553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.535450 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.535426    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-phckq\""
Jul 24 15:47:32.535583 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.535423    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 15:47:32.535702 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.535681    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 15:47:32.535849 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.535827    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 15:47:32.546542 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.546521    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vpr5g"]
Jul 24 15:47:32.548536 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.548515    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 15:47:32.549797 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.549776    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zmcqg"]
Jul 24 15:47:32.549992 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.549972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.551845 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.551824    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 15:47:32.551845 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.551838    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59fhg\""
Jul 24 15:47:32.552165 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.552149    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 15:47:32.552940 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.552915    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68dbdf489c-5rvb6"]
Jul 24 15:47:32.553022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.552974    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vpr5g"]
Jul 24 15:47:32.553022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.553004    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zmcqg"]
Jul 24 15:47:32.553143 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.553125    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.554901 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.554879    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 15:47:32.554986 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.554925    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c9gpx\""
Jul 24 15:47:32.554986 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.554942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 15:47:32.555083 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.554881    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 15:47:32.682834 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.682800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-installation-pull-secrets\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.682992 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.682863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-image-registry-private-configuration\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.682992 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.682935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-bound-sa-token\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.682992 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.682970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14350b95-4ee8-4311-9bf0-f26139c5ca3e-config-volume\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.683142 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-registry-certificates\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.683142 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1690e434-d4c8-40f3-a5bf-0a20714c779a-cert\") pod \"ingress-canary-zmcqg\" (UID: \"1690e434-d4c8-40f3-a5bf-0a20714c779a\") " pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.683142 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwb82\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-kube-api-access-zwb82\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.683286 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44lp\" (UniqueName: \"kubernetes.io/projected/1690e434-d4c8-40f3-a5bf-0a20714c779a-kube-api-access-v44lp\") pod \"ingress-canary-zmcqg\" (UID: \"1690e434-d4c8-40f3-a5bf-0a20714c779a\") " pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.683286 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.683286 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.683286 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14350b95-4ee8-4311-9bf0-f26139c5ca3e-tmp-dir\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.683463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f060f90a-0b0c-48d5-8199-5edf77517497-ca-trust-extracted\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.683463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prn5w\" (UniqueName: \"kubernetes.io/projected/14350b95-4ee8-4311-9bf0-f26139c5ca3e-kube-api-access-prn5w\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.683463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.683399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-trusted-ca\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.687401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.687369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:32.784454 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prn5w\" (UniqueName: \"kubernetes.io/projected/14350b95-4ee8-4311-9bf0-f26139c5ca3e-kube-api-access-prn5w\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-trusted-ca\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-installation-pull-secrets\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-image-registry-private-configuration\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-bound-sa-token\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14350b95-4ee8-4311-9bf0-f26139c5ca3e-config-volume\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-registry-certificates\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1690e434-d4c8-40f3-a5bf-0a20714c779a-cert\") pod \"ingress-canary-zmcqg\" (UID: \"1690e434-d4c8-40f3-a5bf-0a20714c779a\") " pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwb82\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-kube-api-access-zwb82\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.784773 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44lp\" (UniqueName: \"kubernetes.io/projected/1690e434-d4c8-40f3-a5bf-0a20714c779a-kube-api-access-v44lp\") pod \"ingress-canary-zmcqg\" (UID: \"1690e434-d4c8-40f3-a5bf-0a20714c779a\") " pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.785222 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.785222 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.785222 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14350b95-4ee8-4311-9bf0-f26139c5ca3e-tmp-dir\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.785222 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.784914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f060f90a-0b0c-48d5-8199-5edf77517497-ca-trust-extracted\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.785417 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:32.785377    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:47:32.785417 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:32.785396    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:47:32.785509 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:32.785485    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:33.285466375 +0000 UTC m=+35.613030904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:47:32.785909 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.785602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14350b95-4ee8-4311-9bf0-f26139c5ca3e-config-volume\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.785909 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.785818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-registry-certificates\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.785909 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.785859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f060f90a-0b0c-48d5-8199-5edf77517497-ca-trust-extracted\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.785909 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:32.785899    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:47:32.786137 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:32.785952    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:47:33.285932953 +0000 UTC m=+35.613497472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:47:32.786137 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.786000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14350b95-4ee8-4311-9bf0-f26139c5ca3e-tmp-dir\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.786137 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.786019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-trusted-ca\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.790147 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.789767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-image-registry-private-configuration\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.790147 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.789790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-installation-pull-secrets\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.790147 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.790110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1690e434-d4c8-40f3-a5bf-0a20714c779a-cert\") pod \"ingress-canary-zmcqg\" (UID: \"1690e434-d4c8-40f3-a5bf-0a20714c779a\") " pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.796153 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.795676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prn5w\" (UniqueName: \"kubernetes.io/projected/14350b95-4ee8-4311-9bf0-f26139c5ca3e-kube-api-access-prn5w\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:32.796153 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.795879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-bound-sa-token\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.796369 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.796163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwb82\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-kube-api-access-zwb82\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:32.796489 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.796467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44lp\" (UniqueName: \"kubernetes.io/projected/1690e434-d4c8-40f3-a5bf-0a20714c779a-kube-api-access-v44lp\") pod \"ingress-canary-zmcqg\" (UID: \"1690e434-d4c8-40f3-a5bf-0a20714c779a\") " pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:32.797968 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.797864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:32.868943 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:32.868905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zmcqg"
Jul 24 15:47:33.232142 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.232035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:47:33.235948 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.232426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:47:33.235948 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.232547    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:47:33.236291 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.236270    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:47:33.236367 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.236306    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:47:33.236367 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.236351    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:47:33.236468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.236280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k45mm\""
Jul 24 15:47:33.236522 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.236483    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:47:33.237305 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.237285    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w7pwm\""
Jul 24 15:47:33.290075 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.290035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:33.290272 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.290088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:33.290272 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:33.290219    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:47:33.290272 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:33.290242    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:47:33.290408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:33.290297    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:34.290280694 +0000 UTC m=+36.617845210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:47:33.290408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:33.290225    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:47:33.290408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:33.290368    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:47:34.290350638 +0000 UTC m=+36.617915154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:47:33.625390 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:33.625349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zmcqg"]
Jul 24 15:47:33.693828 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:47:33.693793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1690e434_d4c8_40f3_a5bf_0a20714c779a.slice/crio-7b57bc8a7d0d4fb7e1899bf307371a646f3681a9a67e7d1accae46cdbfe8d68f WatchSource:0}: Error finding container 7b57bc8a7d0d4fb7e1899bf307371a646f3681a9a67e7d1accae46cdbfe8d68f: Status 404 returned error can't find the container with id 7b57bc8a7d0d4fb7e1899bf307371a646f3681a9a67e7d1accae46cdbfe8d68f
Jul 24 15:47:34.211972 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.211743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:34.283121 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.283087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:34.296731 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.296702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:34.296731 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.296730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:34.296901 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:34.296826    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:47:34.296901 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:34.296836    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:47:34.296901 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:34.296849    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:47:34.296901 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:34.296875    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:47:36.296861109 +0000 UTC m=+38.624425626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:47:34.296901 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:34.296888    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:36.296881188 +0000 UTC m=+38.624445704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:47:34.342731 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.342698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:34.545245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.545212    2577 generic.go:356] "Generic (PLEG): container finished" podID="97df7b64-4b55-47b8-b30c-956abe38b044" containerID="8f2faf67bfd1f2dcfc9b238834f412e6bd72eaa33501e0647ffa35a327c5a9c6" exitCode=0
Jul 24 15:47:34.545417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.545291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerDied","Data":"8f2faf67bfd1f2dcfc9b238834f412e6bd72eaa33501e0647ffa35a327c5a9c6"}
Jul 24 15:47:34.546760 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:34.546738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zmcqg" event={"ID":"1690e434-d4c8-40f3-a5bf-0a20714c779a","Type":"ContainerStarted","Data":"7b57bc8a7d0d4fb7e1899bf307371a646f3681a9a67e7d1accae46cdbfe8d68f"}
Jul 24 15:47:35.243164 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:35.243125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:35.342706 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:35.342670    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:35.552945 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:35.552909    2577 generic.go:356] "Generic (PLEG): container finished" podID="97df7b64-4b55-47b8-b30c-956abe38b044" containerID="565547a9944b45fd4e2898a2e085d897cf9ea842861448d28244fbc52cf4f6f2" exitCode=0
Jul 24 15:47:35.552945 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:35.552961    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerDied","Data":"565547a9944b45fd4e2898a2e085d897cf9ea842861448d28244fbc52cf4f6f2"}
Jul 24 15:47:36.310771 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.310738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:36.310771 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.310775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:36.311269 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:36.310945    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:47:36.311269 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:36.310967    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:47:36.311269 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:36.310986    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:47:36.311269 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:36.311018    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:47:40.310998055 +0000 UTC m=+42.638562594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:47:36.311269 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:36.311036    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:40.311027658 +0000 UTC m=+42.638592174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:47:36.556968 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.556925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zmcqg" event={"ID":"1690e434-d4c8-40f3-a5bf-0a20714c779a","Type":"ContainerStarted","Data":"9634534e14f60318375ccf99e501186c2e9543f7cf95badc0e90e9f0cec6d803"}
Jul 24 15:47:36.560866 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.560830    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njndp" event={"ID":"97df7b64-4b55-47b8-b30c-956abe38b044","Type":"ContainerStarted","Data":"d81cf76a37703bbea86669a39663ff2b205f336c8660ebb1e737eb1ce8912f5e"}
Jul 24 15:47:36.570957 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.570907    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zmcqg" podStartSLOduration=1.841154334 podStartE2EDuration="4.570889018s" totalImagesPullingTime="2.729734684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:36.570086668 +0000 UTC m=+38.897651206" watchObservedRunningTime="2026-07-24 15:47:36.570889018 +0000 UTC m=+38.898453534"
Jul 24 15:47:36.589529 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.589454    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-njndp" podStartSLOduration=7.855599671 podStartE2EDuration="38.589437834s" totalImagesPullingTime="30.733838163s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:36.588865753 +0000 UTC m=+38.916430298" watchObservedRunningTime="2026-07-24 15:47:36.589437834 +0000 UTC m=+38.917002350"
Jul 24 15:47:36.714149 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.713983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:36.769694 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:36.769584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:37.924260 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:37.924223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:37.973340 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:37.973304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:39.470407 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:39.470368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:40.342563 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:40.342521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:40.342563 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:40.342564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:40.342781 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:40.342724    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:47:40.342781 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:40.342729    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:47:40.342781 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:40.342752    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:47:40.342879 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:40.342787    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:47:48.342772165 +0000 UTC m=+50.670336681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:47:40.342879 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:40.342800    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:48.342794486 +0000 UTC m=+50.670359001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:47:42.700487 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:42.700457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:42.755753 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:42.755720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:46.502055 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:46.502024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:46.562819 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:46.562768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:48.401984 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:48.401944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:47:48.401984 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:48.401984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:47:48.402408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:48.402113    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:47:48.402408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:48.402127    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:47:48.402408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:48.402170    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:47:48.402408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:48.402193    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:04.40217787 +0000 UTC m=+66.729742390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:47:48.402408 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:48.402230    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:48:04.402215486 +0000 UTC m=+66.729780006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:47:48.677039 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:48.676938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:48.777531 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:48.777493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:48.880582 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:48.880548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:48.978196 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:48.978109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:49.314812 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:49.314781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:49.414550 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:49.414519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:49.776709 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:49.776682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:50.155092 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:50.155003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:50.259181 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:50.259146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:50.769558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:50.769514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:50.874034 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:50.873997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:51.697542 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:51.697512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:57.342400 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.342355    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"]
Jul 24 15:47:57.372447 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.372409    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"]
Jul 24 15:47:57.372687 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.372574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.374579 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.374549    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 15:47:57.374758 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.374604    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 15:47:57.375236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.375208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-n9n2j\""
Jul 24 15:47:57.375375 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.375300    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 15:47:57.375375 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.375303    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 15:47:57.386247 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.386222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"]
Jul 24 15:47:57.386370 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.386255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"]
Jul 24 15:47:57.386370 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.386334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:57.388206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.388182    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 24 15:47:57.388372 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.388352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-785jl\""
Jul 24 15:47:57.388696 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.388683    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 24 15:47:57.436824 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.436783    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"]
Jul 24 15:47:57.455344 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.455315    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-msjdg"]
Jul 24 15:47:57.455498 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.455441    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"
Jul 24 15:47:57.461087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.457867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hmb4m\""
Jul 24 15:47:57.463509 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.463479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.463816 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.463792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:57.464009 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.463988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzpq9\" (UniqueName: \"kubernetes.io/projected/e3cfeeb6-3cf6-415c-85cf-71e495eca893-kube-api-access-mzpq9\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.464641 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.464585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c22lt\" (UniqueName: \"kubernetes.io/projected/ccc2c5a0-5645-4d4a-a21e-358bc13fec80-kube-api-access-c22lt\") pod \"network-check-source-7d88cb55d8-zb52t\" (UID: \"ccc2c5a0-5645-4d4a-a21e-358bc13fec80\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"
Jul 24 15:47:57.464759 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.464732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e3cfeeb6-3cf6-415c-85cf-71e495eca893-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.464824 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.464765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:57.480880 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.480820    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-747d9dcfc7-cs2nw"]
Jul 24 15:47:57.481060 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.480956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.482887 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.482847    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 24 15:47:57.482887 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.482884    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 24 15:47:57.483099 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.483029    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 15:47:57.483099 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.483070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pv4fc\""
Jul 24 15:47:57.483176 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.483118    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 15:47:57.489232 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.489207    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 24 15:47:57.498763 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.498718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"]
Jul 24 15:47:57.498950 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.498833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.498950 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.498876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-msjdg"]
Jul 24 15:47:57.498950 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.498906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-747d9dcfc7-cs2nw"]
Jul 24 15:47:57.500727 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.500696    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 24 15:47:57.500997 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.500970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 24 15:47:57.501105 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.501037    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 15:47:57.501105 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.501042    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 24 15:47:57.501178 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.501120    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 15:47:57.501369 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.501357    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z57hc\""
Jul 24 15:47:57.501441 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.501426    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 24 15:47:57.565481 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.565481 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-default-certificate\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.565742 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.565742 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzpq9\" (UniqueName: \"kubernetes.io/projected/e3cfeeb6-3cf6-415c-85cf-71e495eca893-kube-api-access-mzpq9\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.565742 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.565593    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 15:47:57.565742 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c22lt\" (UniqueName: \"kubernetes.io/projected/ccc2c5a0-5645-4d4a-a21e-358bc13fec80-kube-api-access-c22lt\") pod \"network-check-source-7d88cb55d8-zb52t\" (UID: \"ccc2c5a0-5645-4d4a-a21e-358bc13fec80\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"
Jul 24 15:47:57.565742 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.565682    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls podName:e3cfeeb6-3cf6-415c-85cf-71e495eca893 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:58.065664934 +0000 UTC m=+60.393229451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cdps2" (UID: "e3cfeeb6-3cf6-415c-85cf-71e495eca893") : secret "cluster-monitoring-operator-tls" not found
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e3cfeeb6-3cf6-415c-85cf-71e495eca893-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-serving-cert\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6f76\" (UniqueName: \"kubernetes.io/projected/e1a4672e-a764-4839-82de-f0a45a3ed786-kube-api-access-s6f76\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.565959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-stats-auth\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.566493 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.565980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-service-ca-bundle\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.566493 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.566005    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 15:47:57.566493 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.566049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-tmp\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.566493 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.566077    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert podName:f69a5ff3-fda7-4f1d-bf17-88f8f8085399 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:58.066062163 +0000 UTC m=+60.393626680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ndwsn" (UID: "f69a5ff3-fda7-4f1d-bf17-88f8f8085399") : secret "networking-console-plugin-cert" not found
Jul 24 15:47:57.566493 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.566102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-snapshots\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.566493 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.566143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpxlp\" (UniqueName: \"kubernetes.io/projected/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-kube-api-access-vpxlp\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.566774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.566598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:57.566774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.566631    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e3cfeeb6-3cf6-415c-85cf-71e495eca893-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.577551 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.577522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c22lt\" (UniqueName: \"kubernetes.io/projected/ccc2c5a0-5645-4d4a-a21e-358bc13fec80-kube-api-access-c22lt\") pod \"network-check-source-7d88cb55d8-zb52t\" (UID: \"ccc2c5a0-5645-4d4a-a21e-358bc13fec80\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"
Jul 24 15:47:57.577706 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.577522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzpq9\" (UniqueName: \"kubernetes.io/projected/e3cfeeb6-3cf6-415c-85cf-71e495eca893-kube-api-access-mzpq9\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:57.583149 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.583112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:57.638759 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.638667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:57.666881 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.666841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-default-certificate\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.666881 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.666883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.666970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.666993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-serving-cert\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.667057    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6f76\" (UniqueName: \"kubernetes.io/projected/e1a4672e-a764-4839-82de-f0a45a3ed786-kube-api-access-s6f76\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-stats-auth\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-service-ca-bundle\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.667154 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.667156    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:58.16713051 +0000 UTC m=+60.494695040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : secret "router-metrics-certs-default" not found
Jul 24 15:47:57.667573 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-tmp\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.667573 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:57.667239    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:58.167213889 +0000 UTC m=+60.494778417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : configmap references non-existent config key: service-ca.crt
Jul 24 15:47:57.667573 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-snapshots\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.667573 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpxlp\" (UniqueName: \"kubernetes.io/projected/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-kube-api-access-vpxlp\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.667930 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.667867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-service-ca-bundle\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.668170 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.668147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-tmp\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.668269 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.668234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-snapshots\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.668632 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.668592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.669791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.669764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-serving-cert\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.669791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.669786    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-default-certificate\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.669995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.669977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-stats-auth\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.676067 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.676040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6f76\" (UniqueName: \"kubernetes.io/projected/e1a4672e-a764-4839-82de-f0a45a3ed786-kube-api-access-s6f76\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:57.676162 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.676105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpxlp\" (UniqueName: \"kubernetes.io/projected/70f17e22-92d6-46f7-b95f-1ffc1fd2731b-kube-api-access-vpxlp\") pod \"insights-operator-5885fdb759-msjdg\" (UID: \"70f17e22-92d6-46f7-b95f-1ffc1fd2731b\") " pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.767746 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.767699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"
Jul 24 15:47:57.791568 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.791538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-msjdg"
Jul 24 15:47:57.935778 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.935737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t"]
Jul 24 15:47:57.939835 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:47:57.939804    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc2c5a0_5645_4d4a_a21e_358bc13fec80.slice/crio-1b214b5cc4954953686f297672916903cf56859b373c8229d87304602604514b WatchSource:0}: Error finding container 1b214b5cc4954953686f297672916903cf56859b373c8229d87304602604514b: Status 404 returned error can't find the container with id 1b214b5cc4954953686f297672916903cf56859b373c8229d87304602604514b
Jul 24 15:47:57.957330 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:57.957299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-msjdg"]
Jul 24 15:47:57.961285 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:47:57.961257    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f17e22_92d6_46f7_b95f_1ffc1fd2731b.slice/crio-e48b59bcce5be474802fa986df35d8b469b73b2d8b82411cf43e0f05b3b9bccf WatchSource:0}: Error finding container e48b59bcce5be474802fa986df35d8b469b73b2d8b82411cf43e0f05b3b9bccf: Status 404 returned error can't find the container with id e48b59bcce5be474802fa986df35d8b469b73b2d8b82411cf43e0f05b3b9bccf
Jul 24 15:47:58.070267 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.070233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:58.070415 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.070295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:58.070415 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.070371    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 15:47:58.070415 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.070410    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 15:47:58.070536 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.070431    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert podName:f69a5ff3-fda7-4f1d-bf17-88f8f8085399 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:59.070413742 +0000 UTC m=+61.397978261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ndwsn" (UID: "f69a5ff3-fda7-4f1d-bf17-88f8f8085399") : secret "networking-console-plugin-cert" not found
Jul 24 15:47:58.070536 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.070449    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls podName:e3cfeeb6-3cf6-415c-85cf-71e495eca893 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:59.070438566 +0000 UTC m=+61.398003081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cdps2" (UID: "e3cfeeb6-3cf6-415c-85cf-71e495eca893") : secret "cluster-monitoring-operator-tls" not found
Jul 24 15:47:58.170846 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.170754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:58.170846 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.170869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:58.171121 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.170921    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:59.170903586 +0000 UTC m=+61.498468101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : configmap references non-existent config key: service-ca.crt
Jul 24 15:47:58.171121 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.170979    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 15:47:58.171121 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:58.171032    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:47:59.171016812 +0000 UTC m=+61.498581345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : secret "router-metrics-certs-default" not found
Jul 24 15:47:58.535851 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.535822    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgg47"
Jul 24 15:47:58.612187 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.612105    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-msjdg" event={"ID":"70f17e22-92d6-46f7-b95f-1ffc1fd2731b","Type":"ContainerStarted","Data":"e48b59bcce5be474802fa986df35d8b469b73b2d8b82411cf43e0f05b3b9bccf"}
Jul 24 15:47:58.613530 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.613500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t" event={"ID":"ccc2c5a0-5645-4d4a-a21e-358bc13fec80","Type":"ContainerStarted","Data":"1b214b5cc4954953686f297672916903cf56859b373c8229d87304602604514b"}
Jul 24 15:47:58.684172 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.684092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:58.739461 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:58.739420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:59.078408 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:59.078372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:47:59.078676 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:59.078482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:47:59.078676 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.078548    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 15:47:59.078676 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.078589    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 15:47:59.078864 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.078698    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls podName:e3cfeeb6-3cf6-415c-85cf-71e495eca893 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:01.078600122 +0000 UTC m=+63.406164657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cdps2" (UID: "e3cfeeb6-3cf6-415c-85cf-71e495eca893") : secret "cluster-monitoring-operator-tls" not found
Jul 24 15:47:59.078864 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.078725    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert podName:f69a5ff3-fda7-4f1d-bf17-88f8f8085399 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:01.078715064 +0000 UTC m=+63.406279586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ndwsn" (UID: "f69a5ff3-fda7-4f1d-bf17-88f8f8085399") : secret "networking-console-plugin-cert" not found
Jul 24 15:47:59.179453 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:59.179407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:59.179658 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.179580    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 15:47:59.179718 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:47:59.179580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:47:59.179718 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.179673    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:01.179652274 +0000 UTC m=+63.507216834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : secret "router-metrics-certs-default" not found
Jul 24 15:47:59.179718 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:47:59.179702    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:01.179689964 +0000 UTC m=+63.507254481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : configmap references non-existent config key: service-ca.crt
Jul 24 15:48:00.620406 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.620337    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-msjdg" event={"ID":"70f17e22-92d6-46f7-b95f-1ffc1fd2731b","Type":"ContainerStarted","Data":"8a750ca28e55548c52403f685f4ae09c3f587462ecabcacea9289b52d8e2b64a"}
Jul 24 15:48:00.635556 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.635504    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-msjdg" podStartSLOduration=1.783537485 podStartE2EDuration="3.635488706s" totalImagesPullingTime="1.851951221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:00.634792243 +0000 UTC m=+62.962356785" watchObservedRunningTime="2026-07-24 15:48:00.635488706 +0000 UTC m=+62.963053244"
Jul 24 15:48:00.866025 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.865992    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pct5v"]
Jul 24 15:48:00.871634 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.871266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.874219 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.873800    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 15:48:00.874219 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.873923    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qb9zg\""
Jul 24 15:48:00.876113 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.876089    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 15:48:00.890175 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.890134    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pct5v"]
Jul 24 15:48:00.896977 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.896949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11be8611-517c-405e-a8a3-4012c59d082a-data-volume\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.897106 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.896987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.897106 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.897052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11be8611-517c-405e-a8a3-4012c59d082a-crio-socket\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.897106 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.897069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk959\" (UniqueName: \"kubernetes.io/projected/11be8611-517c-405e-a8a3-4012c59d082a-kube-api-access-wk959\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.897273 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.897159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.934337 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.934307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:00.998202 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11be8611-517c-405e-a8a3-4012c59d082a-crio-socket\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.998355 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk959\" (UniqueName: \"kubernetes.io/projected/11be8611-517c-405e-a8a3-4012c59d082a-kube-api-access-wk959\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.998355 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.998355 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11be8611-517c-405e-a8a3-4012c59d082a-crio-socket\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.998498 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:00.998385    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:48:00.998498 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11be8611-517c-405e-a8a3-4012c59d082a-data-volume\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.998498 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:00.998498 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:00.998458    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:01.498439187 +0000 UTC m=+63.826003712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : configmap "kube-rbac-proxy" not found
Jul 24 15:48:00.998713 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:00.998515    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:00.998713 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:00.998574    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:01.498557502 +0000 UTC m=+63.826122018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:00.998882 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:00.998866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11be8611-517c-405e-a8a3-4012c59d082a-data-volume\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:01.008794 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.008768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk959\" (UniqueName: \"kubernetes.io/projected/11be8611-517c-405e-a8a3-4012c59d082a-kube-api-access-wk959\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:01.049038 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.048998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:01.099017 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.098980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:48:01.099183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.099063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:48:01.099183 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.099091    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 15:48:01.099183 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.099156    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert podName:f69a5ff3-fda7-4f1d-bf17-88f8f8085399 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:05.099138218 +0000 UTC m=+67.426702743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ndwsn" (UID: "f69a5ff3-fda7-4f1d-bf17-88f8f8085399") : secret "networking-console-plugin-cert" not found
Jul 24 15:48:01.099483 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.099214    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 15:48:01.099483 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.099292    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls podName:e3cfeeb6-3cf6-415c-85cf-71e495eca893 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:05.099274342 +0000 UTC m=+67.426838867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cdps2" (UID: "e3cfeeb6-3cf6-415c-85cf-71e495eca893") : secret "cluster-monitoring-operator-tls" not found
Jul 24 15:48:01.108667 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.108637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:01.199699 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.199669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:01.199836 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.199778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:01.199895 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.199840    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 15:48:01.199946 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.199900    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:05.199879594 +0000 UTC m=+67.527444129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : configmap references non-existent config key: service-ca.crt
Jul 24 15:48:01.199946 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.199927    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:05.199917363 +0000 UTC m=+67.527481882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : secret "router-metrics-certs-default" not found
Jul 24 15:48:01.501271 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.501201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:01.501403 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.501309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:01.501403 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.501322    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:48:01.501403 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.501383    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:02.501366122 +0000 UTC m=+64.828930638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : configmap "kube-rbac-proxy" not found
Jul 24 15:48:01.501506 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.501420    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:01.501506 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:01.501462    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:02.50145079 +0000 UTC m=+64.829015320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:01.623933 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.623887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t" event={"ID":"ccc2c5a0-5645-4d4a-a21e-358bc13fec80","Type":"ContainerStarted","Data":"aa05b64247f19dc13a492e4a258f18b9a3bbbab57f9f4f52ec71e5cbc55abafb"}
Jul 24 15:48:01.640204 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:01.640152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zb52t" podStartSLOduration=1.462547392 podStartE2EDuration="4.640134742s" totalImagesPullingTime="3.17758735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:01.639675562 +0000 UTC m=+63.967240101" watchObservedRunningTime="2026-07-24 15:48:01.640134742 +0000 UTC m=+63.967699281"
Jul 24 15:48:02.336605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.336574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:02.437165 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.437126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:02.509987 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.509947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:02.510179 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.510065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:02.510179 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:02.510089    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:02.510179 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:02.510168    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:48:02.510304 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:02.510163    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:04.510147119 +0000 UTC m=+66.837711640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:02.510304 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:02.510247    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:04.510227226 +0000 UTC m=+66.837791747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : configmap "kube-rbac-proxy" not found
Jul 24 15:48:02.674354 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.674279    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b2tj6_8105f600-365c-42f2-bd83-2dc9190a4870/dns-node-resolver/0.log"
Jul 24 15:48:02.913708 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.913669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:48:02.913891 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.913750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:48:02.915729 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.915709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:48:02.915793 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.915776    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:48:02.924220 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:02.924202    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 15:48:02.924276 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:02.924259    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs podName:a854e3ef-7cf7-467e-896b-339ed5f3acc6 nodeName:}" failed. No retries permitted until 2026-07-24 15:49:06.924239659 +0000 UTC m=+129.251804177 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs") pod "network-metrics-daemon-pc2tq" (UID: "a854e3ef-7cf7-467e-896b-339ed5f3acc6") : secret "metrics-daemon-secret" not found
Jul 24 15:48:02.926373 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.926331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95304c49-01ee-4b56-8b45-e33b3f2d5b39-original-pull-secret\") pod \"global-pull-secret-syncer-lprwc\" (UID: \"95304c49-01ee-4b56-8b45-e33b3f2d5b39\") " pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:48:02.953602 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:02.953578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lprwc"
Jul 24 15:48:03.015256 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.015038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:48:03.018459 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.018433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpp76\" (UniqueName: \"kubernetes.io/projected/a2e14ee3-4a59-4833-bab8-73545f677c9c-kube-api-access-qpp76\") pod \"network-check-target-swp2x\" (UID: \"a2e14ee3-4a59-4833-bab8-73545f677c9c\") " pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:48:03.071718 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.071690    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lprwc"]
Jul 24 15:48:03.076013 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:03.075990    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95304c49_01ee_4b56_8b45_e33b3f2d5b39.slice/crio-3f9074df0cb4585ceaaa3a60c6fdd513380e8183c0d92c3fd24a6aead14ffa2f WatchSource:0}: Error finding container 3f9074df0cb4585ceaaa3a60c6fdd513380e8183c0d92c3fd24a6aead14ffa2f: Status 404 returned error can't find the container with id 3f9074df0cb4585ceaaa3a60c6fdd513380e8183c0d92c3fd24a6aead14ffa2f
Jul 24 15:48:03.214957 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.214881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:03.260805 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.260783    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w7pwm\""
Jul 24 15:48:03.270081 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.270062    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:48:03.316638 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.316600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:03.416858 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.416752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-swp2x"]
Jul 24 15:48:03.417897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.417870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:03.419402 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:03.419381    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e14ee3_4a59_4833_bab8_73545f677c9c.slice/crio-e52ffe0eeb4c0c8dec0ab78ff68460e53320d655c874825ab6f436df54d9d331 WatchSource:0}: Error finding container e52ffe0eeb4c0c8dec0ab78ff68460e53320d655c874825ab6f436df54d9d331: Status 404 returned error can't find the container with id e52ffe0eeb4c0c8dec0ab78ff68460e53320d655c874825ab6f436df54d9d331
Jul 24 15:48:03.518055 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.518030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:03.629842 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.629802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lprwc" event={"ID":"95304c49-01ee-4b56-8b45-e33b3f2d5b39","Type":"ContainerStarted","Data":"3f9074df0cb4585ceaaa3a60c6fdd513380e8183c0d92c3fd24a6aead14ffa2f"}
Jul 24 15:48:03.631106 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.631083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-swp2x" event={"ID":"a2e14ee3-4a59-4833-bab8-73545f677c9c","Type":"ContainerStarted","Data":"84c08631aff3439cc36f0da6be3da26a7035c048994ea33e5efde3051e717646"}
Jul 24 15:48:03.631218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.631124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-swp2x" event={"ID":"a2e14ee3-4a59-4833-bab8-73545f677c9c","Type":"ContainerStarted","Data":"e52ffe0eeb4c0c8dec0ab78ff68460e53320d655c874825ab6f436df54d9d331"}
Jul 24 15:48:03.631274 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.631212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:48:03.646510 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:03.646459    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-swp2x" podStartSLOduration=65.646445369 podStartE2EDuration="1m5.646445369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:03.645655554 +0000 UTC m=+65.973220086" watchObservedRunningTime="2026-07-24 15:48:03.646445369 +0000 UTC m=+65.974009908"
Jul 24 15:48:04.078233 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.078184    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lbzkd_9043edc6-8df3-47b5-9318-b7a478366ec3/node-ca/0.log"
Jul 24 15:48:04.097487 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.097458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:04.203696 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.203664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:04.298708 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.298675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:04.351699 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.351632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:04.426068 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.426036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:48:04.426201 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.426144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:48:04.426201 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.426182    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:48:04.426290 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.426242    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls podName:14350b95-4ee8-4311-9bf0-f26139c5ca3e nodeName:}" failed. No retries permitted until 2026-07-24 15:48:36.426225025 +0000 UTC m=+98.753789563 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls") pod "dns-default-vpr5g" (UID: "14350b95-4ee8-4311-9bf0-f26139c5ca3e") : secret "dns-default-metrics-tls" not found
Jul 24 15:48:04.426290 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.426277    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:48:04.430373 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.426541    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68dbdf489c-5rvb6: secret "image-registry-tls" not found
Jul 24 15:48:04.430373 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.426651    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls podName:f060f90a-0b0c-48d5-8199-5edf77517497 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:36.42660744 +0000 UTC m=+98.754171973 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls") pod "image-registry-68dbdf489c-5rvb6" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497") : secret "image-registry-tls" not found
Jul 24 15:48:04.527060 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.527022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:04.527184 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.527122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:04.527184 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.527167    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:48:04.527273 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.527236    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:08.527217643 +0000 UTC m=+70.854782164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : configmap "kube-rbac-proxy" not found
Jul 24 15:48:04.527273 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.527242    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:04.527364 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:04.527284    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:08.527273044 +0000 UTC m=+70.854837575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:04.874824 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:04.874795    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zmcqg_1690e434-d4c8-40f3-a5bf-0a20714c779a/serve-healthcheck-canary/0.log"
Jul 24 15:48:05.132302 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.132223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:48:05.132767 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.132365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:48:05.132767 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.132371    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 15:48:05.132767 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.132433    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls podName:e3cfeeb6-3cf6-415c-85cf-71e495eca893 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:13.132417186 +0000 UTC m=+75.459981702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cdps2" (UID: "e3cfeeb6-3cf6-415c-85cf-71e495eca893") : secret "cluster-monitoring-operator-tls" not found
Jul 24 15:48:05.132767 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.132512    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 15:48:05.132767 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.132574    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert podName:f69a5ff3-fda7-4f1d-bf17-88f8f8085399 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:13.132552983 +0000 UTC m=+75.460117499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ndwsn" (UID: "f69a5ff3-fda7-4f1d-bf17-88f8f8085399") : secret "networking-console-plugin-cert" not found
Jul 24 15:48:05.233831 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.233797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:05.233986 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.233927    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 15:48:05.233986 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.233969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:05.234078 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.233988    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:13.233970172 +0000 UTC m=+75.561534699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : secret "router-metrics-certs-default" not found
Jul 24 15:48:05.234078 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:05.234064    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:13.234047904 +0000 UTC m=+75.561612424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : configmap references non-existent config key: service-ca.crt
Jul 24 15:48:05.739164 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.739129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:05.839904 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.839869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:05.939971 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:05.939925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:06.297123 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:06.297071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:06.363664 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:06.363599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:06.579748 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:06.579635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:07.643185 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:07.643153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lprwc" event={"ID":"95304c49-01ee-4b56-8b45-e33b3f2d5b39","Type":"ContainerStarted","Data":"d7a95ae68340e20bfc88c23c0739779e46b0634eb1b73ceba848d3c33c13f00a"}
Jul 24 15:48:07.658052 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:07.658008    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lprwc" podStartSLOduration=65.726160877 podStartE2EDuration="1m9.657995669s" totalImagesPullingTime="3.931834792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:07.657385385 +0000 UTC m=+69.984949923" watchObservedRunningTime="2026-07-24 15:48:07.657995669 +0000 UTC m=+69.985560206"
Jul 24 15:48:08.560091 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:08.560056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:08.560259 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:08.560130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:08.560259 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:08.560178    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:48:08.560259 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:08.560216    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:08.560259 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:08.560237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:16.560223207 +0000 UTC m=+78.887787723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : configmap "kube-rbac-proxy" not found
Jul 24 15:48:08.560395 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:08.560263    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:16.560250355 +0000 UTC m=+78.887814871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:09.812631 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:09.812581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:10.752454 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:10.752416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:10.851959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:10.851931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:10.952686 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:10.952653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:11.152178 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:11.152150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:11.252874 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:11.252826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:11.388662 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:11.388629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:12.397084 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:12.397035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:12.592126 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:12.592087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:12.985078 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:12.985042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:13.096659 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.096627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:13.189326 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.189294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:13.194325 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.194302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:48:13.194470 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.194452    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 15:48:13.194512 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.194484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:48:13.194558 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.194532    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert podName:f69a5ff3-fda7-4f1d-bf17-88f8f8085399 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:29.194513898 +0000 UTC m=+91.522078419 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-ndwsn" (UID: "f69a5ff3-fda7-4f1d-bf17-88f8f8085399") : secret "networking-console-plugin-cert" not found
Jul 24 15:48:13.194644 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.194567    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 15:48:13.194644 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.194610    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls podName:e3cfeeb6-3cf6-415c-85cf-71e495eca893 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:29.194597696 +0000 UTC m=+91.522162226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cdps2" (UID: "e3cfeeb6-3cf6-415c-85cf-71e495eca893") : secret "cluster-monitoring-operator-tls" not found
Jul 24 15:48:13.295064 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.295032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:13.295225 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.295143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:13.295225 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.295187    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:29.295170576 +0000 UTC m=+91.622735092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : configmap references non-existent config key: service-ca.crt
Jul 24 15:48:13.295357 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.295267    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 15:48:13.295357 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:13.295337    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs podName:e1a4672e-a764-4839-82de-f0a45a3ed786 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:29.295319322 +0000 UTC m=+91.622883843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs") pod "router-default-747d9dcfc7-cs2nw" (UID: "e1a4672e-a764-4839-82de-f0a45a3ed786") : secret "router-metrics-certs-default" not found
Jul 24 15:48:13.785871 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:13.785837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:14.107290 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:14.107208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:14.208076 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:14.208043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:14.385469 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:14.385381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:14.592919 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:14.592880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:14.645939 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:14.645865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:14.990251 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:14.990170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:15.188629 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:15.188589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:15.276005 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:15.275978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:15.785239 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:15.785209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:16.187095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.187012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:16.587053 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.587020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:16.622468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.622444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:16.622605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.622514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:16.622667 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:16.622646    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:16.622714 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:16.622704    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls podName:11be8611-517c-405e-a8a3-4012c59d082a nodeName:}" failed. No retries permitted until 2026-07-24 15:48:32.622687203 +0000 UTC m=+94.950251738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pct5v" (UID: "11be8611-517c-405e-a8a3-4012c59d082a") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:48:16.622942 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.622924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11be8611-517c-405e-a8a3-4012c59d082a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:16.896340 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.896254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:16.996761 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:16.996727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:17.429092 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:17.429053    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:17.590448 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:17.590413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:17.984655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:17.984624    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:18.790949 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:18.790913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.008935 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:19.008905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.385425 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:19.385391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.588040 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:19.588008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.788757 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:19.788727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:20.587990 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:20.587954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:20.990191 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:20.990103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:21.187649 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:21.187601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:21.312786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:21.312751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:21.416028 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:21.415997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:21.470325 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:21.470294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:21.612212 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:21.612119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:21.986019 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:21.985932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:22.194331 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:22.194302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:22.257248 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:22.257166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:22.811929 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:22.811898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:22.866123 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:22.866081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:23.021099 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:23.021066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:23.189671 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:23.189562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:23.414048 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:23.414008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:25.093926 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.093878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:25.230119 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.230086    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-5vjb8"]
Jul 24 15:48:25.234342 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.234325    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:25.236605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.236576    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 15:48:25.236605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.236593    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 15:48:25.236817 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.236603    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-swwl2\""
Jul 24 15:48:25.236903 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.236870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:25.244422 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.244399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-5vjb8"]
Jul 24 15:48:25.290954 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.290931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpgdw\" (UniqueName: \"kubernetes.io/projected/73956ad7-24a7-40f8-9fa3-40900fc523f8-kube-api-access-gpgdw\") pod \"downloads-7c97dc4555-5vjb8\" (UID: \"73956ad7-24a7-40f8-9fa3-40900fc523f8\") " pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:25.325922 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.325899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:25.391729 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.391666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpgdw\" (UniqueName: \"kubernetes.io/projected/73956ad7-24a7-40f8-9fa3-40900fc523f8-kube-api-access-gpgdw\") pod \"downloads-7c97dc4555-5vjb8\" (UID: \"73956ad7-24a7-40f8-9fa3-40900fc523f8\") " pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:25.401760 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.401739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpgdw\" (UniqueName: \"kubernetes.io/projected/73956ad7-24a7-40f8-9fa3-40900fc523f8-kube-api-access-gpgdw\") pod \"downloads-7c97dc4555-5vjb8\" (UID: \"73956ad7-24a7-40f8-9fa3-40900fc523f8\") " pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:25.423865 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.423826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:25.524547 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.524512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:25.542754 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.542721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:25.682599 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:25.682537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-5vjb8"]
Jul 24 15:48:25.685306 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:25.685276    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73956ad7_24a7_40f8_9fa3_40900fc523f8.slice/crio-d8b6c8be7d1e2b457d68a6160c0df950a348761d3b8f87576aeedd58372a99e8 WatchSource:0}: Error finding container d8b6c8be7d1e2b457d68a6160c0df950a348761d3b8f87576aeedd58372a99e8: Status 404 returned error can't find the container with id d8b6c8be7d1e2b457d68a6160c0df950a348761d3b8f87576aeedd58372a99e8
Jul 24 15:48:26.689265 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:26.689214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-5vjb8" event={"ID":"73956ad7-24a7-40f8-9fa3-40900fc523f8","Type":"ContainerStarted","Data":"d8b6c8be7d1e2b457d68a6160c0df950a348761d3b8f87576aeedd58372a99e8"}
Jul 24 15:48:26.784138 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:26.784090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:26.942179 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:26.942063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:27.004439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:27.004403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:27.703516 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:27.703470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:27.806057 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:27.805996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:29.226423 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.226381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:48:29.226905 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.226473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:48:29.229164 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.229137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f69a5ff3-fda7-4f1d-bf17-88f8f8085399-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ndwsn\" (UID: \"f69a5ff3-fda7-4f1d-bf17-88f8f8085399\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:48:29.229296 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.229137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3cfeeb6-3cf6-415c-85cf-71e495eca893-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cdps2\" (UID: \"e3cfeeb6-3cf6-415c-85cf-71e495eca893\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:48:29.326990 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.326959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:29.327157 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.327066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:29.327697 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.327666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a4672e-a764-4839-82de-f0a45a3ed786-service-ca-bundle\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:29.329405 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.329380    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1a4672e-a764-4839-82de-f0a45a3ed786-metrics-certs\") pod \"router-default-747d9dcfc7-cs2nw\" (UID: \"e1a4672e-a764-4839-82de-f0a45a3ed786\") " pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:29.483368 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.483282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-n9n2j\""
Jul 24 15:48:29.491790 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.491764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"
Jul 24 15:48:29.497598 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.497410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-785jl\""
Jul 24 15:48:29.506927 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.506739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"
Jul 24 15:48:29.610953 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.610916    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z57hc\""
Jul 24 15:48:29.619533 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.619508    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:29.633882 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.633625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2"]
Jul 24 15:48:29.636048 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:29.636019    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cfeeb6_3cf6_415c_85cf_71e495eca893.slice/crio-94fbf7e122e8c793385c5f6bde152ab3e65367103ac87b3080de8a48776e49a8 WatchSource:0}: Error finding container 94fbf7e122e8c793385c5f6bde152ab3e65367103ac87b3080de8a48776e49a8: Status 404 returned error can't find the container with id 94fbf7e122e8c793385c5f6bde152ab3e65367103ac87b3080de8a48776e49a8
Jul 24 15:48:29.655886 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.655860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn"]
Jul 24 15:48:29.659339 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:29.659288    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69a5ff3_fda7_4f1d_bf17_88f8f8085399.slice/crio-7baaad101670c88288548620436cba5fa2902b5b283a1f21332f54b3066584d9 WatchSource:0}: Error finding container 7baaad101670c88288548620436cba5fa2902b5b283a1f21332f54b3066584d9: Status 404 returned error can't find the container with id 7baaad101670c88288548620436cba5fa2902b5b283a1f21332f54b3066584d9
Jul 24 15:48:29.699481 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.699416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn" event={"ID":"f69a5ff3-fda7-4f1d-bf17-88f8f8085399","Type":"ContainerStarted","Data":"7baaad101670c88288548620436cba5fa2902b5b283a1f21332f54b3066584d9"}
Jul 24 15:48:29.700704 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.700645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2" event={"ID":"e3cfeeb6-3cf6-415c-85cf-71e495eca893","Type":"ContainerStarted","Data":"94fbf7e122e8c793385c5f6bde152ab3e65367103ac87b3080de8a48776e49a8"}
Jul 24 15:48:29.760332 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:29.760258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-747d9dcfc7-cs2nw"]
Jul 24 15:48:29.763248 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:29.763212    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a4672e_a764_4839_82de_f0a45a3ed786.slice/crio-eea311651c3af05aafd53459a4bc2f796bdfa611d1b623736a01b2d937398cd2 WatchSource:0}: Error finding container eea311651c3af05aafd53459a4bc2f796bdfa611d1b623736a01b2d937398cd2: Status 404 returned error can't find the container with id eea311651c3af05aafd53459a4bc2f796bdfa611d1b623736a01b2d937398cd2
Jul 24 15:48:30.709638 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:30.708826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw" event={"ID":"e1a4672e-a764-4839-82de-f0a45a3ed786","Type":"ContainerStarted","Data":"fa1147143712c070b218ecbd8125a3539035e58123006f59d0cf014ccd881043"}
Jul 24 15:48:30.709638 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:30.708897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw" event={"ID":"e1a4672e-a764-4839-82de-f0a45a3ed786","Type":"ContainerStarted","Data":"eea311651c3af05aafd53459a4bc2f796bdfa611d1b623736a01b2d937398cd2"}
Jul 24 15:48:30.728444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:30.728376    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw" podStartSLOduration=33.728355359 podStartE2EDuration="33.728355359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:30.727205228 +0000 UTC m=+93.054769779" watchObservedRunningTime="2026-07-24 15:48:30.728355359 +0000 UTC m=+93.055919899"
Jul 24 15:48:31.620694 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.620655    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:31.623689 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.623665    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:31.714009 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.713963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn" event={"ID":"f69a5ff3-fda7-4f1d-bf17-88f8f8085399","Type":"ContainerStarted","Data":"23c5aac49a207c2ea8ce145c32cf1d6b4b6eb6a55418fef0af0e8bbe417dd746"}
Jul 24 15:48:31.714498 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.714087    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:31.715525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.715502    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-747d9dcfc7-cs2nw"
Jul 24 15:48:31.727792 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.727739    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ndwsn" podStartSLOduration=33.48864871 podStartE2EDuration="34.727722665s" totalImagesPullingTime="1.239073955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:31.727261507 +0000 UTC m=+94.054826066" watchObservedRunningTime="2026-07-24 15:48:31.727722665 +0000 UTC m=+94.055287198"
Jul 24 15:48:31.828752 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.828713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:31.889324 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.889220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:31.954311 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:31.954270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:32.379817 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.379770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:32.484002 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.483955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:32.543529 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.543479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:32.658448 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.658369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:32.661079 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.661056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11be8611-517c-405e-a8a3-4012c59d082a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pct5v\" (UID: \"11be8611-517c-405e-a8a3-4012c59d082a\") " pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:32.683167 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.683123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pct5v"
Jul 24 15:48:32.722872 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.722836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2" event={"ID":"e3cfeeb6-3cf6-415c-85cf-71e495eca893","Type":"ContainerStarted","Data":"8da03d3ff3b84b42e65093fac68d5f5d83daf3d1b98e0c15ffc3fa4478e50082"}
Jul 24 15:48:32.737625 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.737549    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cdps2" podStartSLOduration=33.412105263 podStartE2EDuration="35.737528303s" totalImagesPullingTime="2.32542304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:32.736060736 +0000 UTC m=+95.063625274" watchObservedRunningTime="2026-07-24 15:48:32.737528303 +0000 UTC m=+95.065092842"
Jul 24 15:48:32.855075 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.855042    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pct5v"]
Jul 24 15:48:32.858741 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:32.858705    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11be8611_517c_405e_a8a3_4012c59d082a.slice/crio-4684833b4ba4fad44553c24aaf65eb64e9b134a0efab3d5d5d9056564036bc63 WatchSource:0}: Error finding container 4684833b4ba4fad44553c24aaf65eb64e9b134a0efab3d5d5d9056564036bc63: Status 404 returned error can't find the container with id 4684833b4ba4fad44553c24aaf65eb64e9b134a0efab3d5d5d9056564036bc63
Jul 24 15:48:32.920698 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:32.920594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:33.741347 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:33.741188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pct5v" event={"ID":"11be8611-517c-405e-a8a3-4012c59d082a","Type":"ContainerStarted","Data":"4684833b4ba4fad44553c24aaf65eb64e9b134a0efab3d5d5d9056564036bc63"}
Jul 24 15:48:34.638417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:34.638366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-swp2x"
Jul 24 15:48:34.923519 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:34.923429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:34.988446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:34.988405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:35.048183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.048083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:35.541181 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.541127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:35.585269 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.585225    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"]
Jul 24 15:48:35.590770 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.590739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.593087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.593059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xd698\""
Jul 24 15:48:35.593087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.593059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 15:48:35.593270 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.593098    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 15:48:35.593270 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.593059    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 15:48:35.596743 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.596723    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"]
Jul 24 15:48:35.608896 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.608860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:35.688352 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.688309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1af98115-d18d-4479-a87d-cc8b38c5ef41-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.688529 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.688372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af98115-d18d-4479-a87d-cc8b38c5ef41-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.688529 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.688457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxn8w\" (UniqueName: \"kubernetes.io/projected/1af98115-d18d-4479-a87d-cc8b38c5ef41-kube-api-access-wxn8w\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.688529 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.688497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1af98115-d18d-4479-a87d-cc8b38c5ef41-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.789853 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.789812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1af98115-d18d-4479-a87d-cc8b38c5ef41-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.790040 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.789868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af98115-d18d-4479-a87d-cc8b38c5ef41-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.790040 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.789922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxn8w\" (UniqueName: \"kubernetes.io/projected/1af98115-d18d-4479-a87d-cc8b38c5ef41-kube-api-access-wxn8w\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.790040 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.789957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1af98115-d18d-4479-a87d-cc8b38c5ef41-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.791082 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.791022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af98115-d18d-4479-a87d-cc8b38c5ef41-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.793045 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.792961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1af98115-d18d-4479-a87d-cc8b38c5ef41-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.793225 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.793202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1af98115-d18d-4479-a87d-cc8b38c5ef41-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.797859 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.797829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxn8w\" (UniqueName: \"kubernetes.io/projected/1af98115-d18d-4479-a87d-cc8b38c5ef41-kube-api-access-wxn8w\") pod \"prometheus-operator-84d69f794d-bcnsq\" (UID: \"1af98115-d18d-4479-a87d-cc8b38c5ef41\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:35.808342 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.808316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:35.902410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:35.902365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"
Jul 24 15:48:36.497351 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.497309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:48:36.497761 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.497422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:48:36.499806 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.499781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14350b95-4ee8-4311-9bf0-f26139c5ca3e-metrics-tls\") pod \"dns-default-vpr5g\" (UID: \"14350b95-4ee8-4311-9bf0-f26139c5ca3e\") " pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:48:36.499973 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.499955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"image-registry-68dbdf489c-5rvb6\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") " pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:48:36.746633 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.746585    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-phckq\""
Jul 24 15:48:36.755574 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.755493    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:48:36.764828 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.764802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59fhg\""
Jul 24 15:48:36.773175 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.773133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:48:36.879989 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.879951    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bcf75f55-bcsvp"]
Jul 24 15:48:36.884385 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.884362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:36.886600 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.886573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pb956\""
Jul 24 15:48:36.886733 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.886585    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 15:48:36.886733 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.886717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 15:48:36.886968 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.886945    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 15:48:36.886968 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.886956    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 15:48:36.887293 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.887279    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 15:48:36.898029 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:36.897982    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bcf75f55-bcsvp"]
Jul 24 15:48:37.001909 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.001868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-service-ca\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.002087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.001934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg9wf\" (UniqueName: \"kubernetes.io/projected/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-kube-api-access-pg9wf\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.002087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.002015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-serving-cert\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.002087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.002045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-oauth-serving-cert\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.002258 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.002095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-config\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.002258 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.002140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-oauth-config\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.103698 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.103595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-serving-cert\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.103698 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.103704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-oauth-serving-cert\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.104174 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.103776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-config\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.104174 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.103849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-oauth-config\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.104174 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.103942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-service-ca\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.104174 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.104020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg9wf\" (UniqueName: \"kubernetes.io/projected/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-kube-api-access-pg9wf\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.104895 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.104858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-oauth-serving-cert\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.105050 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.104858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-config\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.105231 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.105198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-service-ca\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.107338 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.107309    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-oauth-config\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.107504 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.107402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-serving-cert\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.113815 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.113730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg9wf\" (UniqueName: \"kubernetes.io/projected/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-kube-api-access-pg9wf\") pod \"console-bcf75f55-bcsvp\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") " pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:37.199557 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:37.199516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:38.021275 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:38.021221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:38.073316 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:38.073283    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:38.467666 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:38.467519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:38.567707 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:38.567663    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:39.852325 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:39.852270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:39.954278 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:39.954213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:43.245548 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.245474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:43.341957 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.341857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:43.623096 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.623059    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8d6694c4d-zw9qt"]
Jul 24 15:48:43.627008 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.626974    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.639348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.639300    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 15:48:43.641659 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.641597    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d6694c4d-zw9qt"]
Jul 24 15:48:43.767994 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.767938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-serving-cert\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.768184 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.768010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-console-config\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.768184 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.768047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-oauth-config\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.768184 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.768083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75sv5\" (UniqueName: \"kubernetes.io/projected/a675627a-3c12-46c5-a633-b22e9afffbd3-kube-api-access-75sv5\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.768361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.768185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-trusted-ca-bundle\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.768361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.768298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-oauth-serving-cert\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.768361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.768346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-service-ca\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.869726 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-serving-cert\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.869920 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-console-config\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.869920 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-oauth-config\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.869920 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75sv5\" (UniqueName: \"kubernetes.io/projected/a675627a-3c12-46c5-a633-b22e9afffbd3-kube-api-access-75sv5\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.869920 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-trusted-ca-bundle\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.869920 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-oauth-serving-cert\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.870203 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.869920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-service-ca\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.870581 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.870547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-console-config\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.870724 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.870602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-service-ca\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.870848 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.870829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-oauth-serving-cert\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.870982 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.870962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-trusted-ca-bundle\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.872908 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.872870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-serving-cert\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.873023 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.872872    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-oauth-config\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.878234 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.878180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75sv5\" (UniqueName: \"kubernetes.io/projected/a675627a-3c12-46c5-a633-b22e9afffbd3-kube-api-access-75sv5\") pod \"console-8d6694c4d-zw9qt\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") " pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:43.945122 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:43.945042    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:48:44.429044 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:44.429006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:44.529488 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:44.529450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:45.212635 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.212550    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bcf75f55-bcsvp"]
Jul 24 15:48:45.215590 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:45.215550    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d76a7a_c501_4c15_b7fb_0949bc6bf4f9.slice/crio-d8c43d623df7e1658c750bb1f69c1e8373257b12ba9c85fdcfc781b3fbc232d7 WatchSource:0}: Error finding container d8c43d623df7e1658c750bb1f69c1e8373257b12ba9c85fdcfc781b3fbc232d7: Status 404 returned error can't find the container with id d8c43d623df7e1658c750bb1f69c1e8373257b12ba9c85fdcfc781b3fbc232d7
Jul 24 15:48:45.282378 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.282315    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68dbdf489c-5rvb6"]
Jul 24 15:48:45.286994 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:45.286963    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf060f90a_0b0c_48d5_8199_5edf77517497.slice/crio-58e63e89b879d12345f2e648be0045fe7ce147f63c21d1b8482e491073677717 WatchSource:0}: Error finding container 58e63e89b879d12345f2e648be0045fe7ce147f63c21d1b8482e491073677717: Status 404 returned error can't find the container with id 58e63e89b879d12345f2e648be0045fe7ce147f63c21d1b8482e491073677717
Jul 24 15:48:45.295923 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.295887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d6694c4d-zw9qt"]
Jul 24 15:48:45.300971 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:45.300933    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda675627a_3c12_46c5_a633_b22e9afffbd3.slice/crio-a5b9a0d5fd268cbfc3376aec3f5562e8ab5bf84c691312f17e17568977be3d65 WatchSource:0}: Error finding container a5b9a0d5fd268cbfc3376aec3f5562e8ab5bf84c691312f17e17568977be3d65: Status 404 returned error can't find the container with id a5b9a0d5fd268cbfc3376aec3f5562e8ab5bf84c691312f17e17568977be3d65
Jul 24 15:48:45.449670 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.449543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-bcnsq"]
Jul 24 15:48:45.452207 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.452181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vpr5g"]
Jul 24 15:48:45.455856 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:45.455824    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af98115_d18d_4479_a87d_cc8b38c5ef41.slice/crio-2686ab55ae3641b03f0d2152854c4bd4236e363747f8de3a187b4ac3f79103ea WatchSource:0}: Error finding container 2686ab55ae3641b03f0d2152854c4bd4236e363747f8de3a187b4ac3f79103ea: Status 404 returned error can't find the container with id 2686ab55ae3641b03f0d2152854c4bd4236e363747f8de3a187b4ac3f79103ea
Jul 24 15:48:45.456549 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:45.456512    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14350b95_4ee8_4311_9bf0_f26139c5ca3e.slice/crio-41475f4b6456d71e0417ae8e30af26914dc892d24858b73944d228bc9a6be406 WatchSource:0}: Error finding container 41475f4b6456d71e0417ae8e30af26914dc892d24858b73944d228bc9a6be406: Status 404 returned error can't find the container with id 41475f4b6456d71e0417ae8e30af26914dc892d24858b73944d228bc9a6be406
Jul 24 15:48:45.615868 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.615716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:45.719682 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.719551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:45.793115 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.793076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpr5g" event={"ID":"14350b95-4ee8-4311-9bf0-f26139c5ca3e","Type":"ContainerStarted","Data":"41475f4b6456d71e0417ae8e30af26914dc892d24858b73944d228bc9a6be406"}
Jul 24 15:48:45.795933 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.795900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-5vjb8" event={"ID":"73956ad7-24a7-40f8-9fa3-40900fc523f8","Type":"ContainerStarted","Data":"6f2641b45ffa9699c7dbf220322499052c69e0ae2832d8d7f378cb00f87889dd"}
Jul 24 15:48:45.796354 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.796333    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:45.800049 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.800019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pct5v" event={"ID":"11be8611-517c-405e-a8a3-4012c59d082a","Type":"ContainerStarted","Data":"d26288a88af890961db3976d933ebeacae119848bf8125c950e67399f564d715"}
Jul 24 15:48:45.802634 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.802589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6" event={"ID":"f060f90a-0b0c-48d5-8199-5edf77517497","Type":"ContainerStarted","Data":"46a2dc00cc99a7760ec59d5c4b7d27842dbb912d14384d55ef2760bf7dc1034c"}
Jul 24 15:48:45.802733 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.802647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6" event={"ID":"f060f90a-0b0c-48d5-8199-5edf77517497","Type":"ContainerStarted","Data":"58e63e89b879d12345f2e648be0045fe7ce147f63c21d1b8482e491073677717"}
Jul 24 15:48:45.803234 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.803182    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:48:45.806338 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.806279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq" event={"ID":"1af98115-d18d-4479-a87d-cc8b38c5ef41","Type":"ContainerStarted","Data":"2686ab55ae3641b03f0d2152854c4bd4236e363747f8de3a187b4ac3f79103ea"}
Jul 24 15:48:45.808774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.808711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-5vjb8"
Jul 24 15:48:45.812011 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.811900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bcf75f55-bcsvp" event={"ID":"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9","Type":"ContainerStarted","Data":"d8c43d623df7e1658c750bb1f69c1e8373257b12ba9c85fdcfc781b3fbc232d7"}
Jul 24 15:48:45.815519 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.814317    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-5vjb8" podStartSLOduration=1.424097623 podStartE2EDuration="20.814301372s" totalImagesPullingTime="19.390203749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:45.811562407 +0000 UTC m=+108.139126948" watchObservedRunningTime="2026-07-24 15:48:45.814301372 +0000 UTC m=+108.141865914"
Jul 24 15:48:45.817407 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.817370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d6694c4d-zw9qt" event={"ID":"a675627a-3c12-46c5-a633-b22e9afffbd3","Type":"ContainerStarted","Data":"a5b9a0d5fd268cbfc3376aec3f5562e8ab5bf84c691312f17e17568977be3d65"}
Jul 24 15:48:45.846849 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:45.846755    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6" podStartSLOduration=107.846719466 podStartE2EDuration="1m47.846719466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:45.841961623 +0000 UTC m=+108.169526163" watchObservedRunningTime="2026-07-24 15:48:45.846719466 +0000 UTC m=+108.174284006"
Jul 24 15:48:47.386691 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:47.386481    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68dbdf489c-5rvb6"]
Jul 24 15:48:47.464651 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:47.464513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:47.562058 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:47.562005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:47.617188 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:47.617128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:51.081993 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:51.081939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:51.136336 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:51.136299    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:52.838975 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:52.838937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:54.699465 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.699417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:54.883547 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.883462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d6694c4d-zw9qt" event={"ID":"a675627a-3c12-46c5-a633-b22e9afffbd3","Type":"ContainerStarted","Data":"da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4"}
Jul 24 15:48:54.886339 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.886307    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpr5g" event={"ID":"14350b95-4ee8-4311-9bf0-f26139c5ca3e","Type":"ContainerStarted","Data":"6ad31608e7a4f4863c294972fedc506a32ecf2ea91aa7e0f6ad60d919c622531"}
Jul 24 15:48:54.886478 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.886406    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:48:54.886478 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.886440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpr5g" event={"ID":"14350b95-4ee8-4311-9bf0-f26139c5ca3e","Type":"ContainerStarted","Data":"6876c26d40d94ba9f6eb8c5a1d9772162c74b0c581920d8f6020450b6d203a11"}
Jul 24 15:48:54.888857 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.888826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pct5v" event={"ID":"11be8611-517c-405e-a8a3-4012c59d082a","Type":"ContainerStarted","Data":"8a5431895a9b447ec62837cafdddaea93c4accd4d09d9f38722a98cd529fce00"}
Jul 24 15:48:54.891247 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.891221    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq" event={"ID":"1af98115-d18d-4479-a87d-cc8b38c5ef41","Type":"ContainerStarted","Data":"6a9c400f27791141b76eb8f4ce9f3ea4bc73ffdc663bc27bf390a38eb2497ecf"}
Jul 24 15:48:54.891375 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.891274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq" event={"ID":"1af98115-d18d-4479-a87d-cc8b38c5ef41","Type":"ContainerStarted","Data":"5d0af882ea5768524296415b653f8e22cb985770af90a516cb8bf66830bc4c1f"}
Jul 24 15:48:54.892826 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.892805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bcf75f55-bcsvp" event={"ID":"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9","Type":"ContainerStarted","Data":"6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3"}
Jul 24 15:48:54.901429 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.901391    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8d6694c4d-zw9qt" podStartSLOduration=2.835894094 podStartE2EDuration="11.901379083s" totalImagesPullingTime="9.065484989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:54.899891922 +0000 UTC m=+117.227456461" watchObservedRunningTime="2026-07-24 15:48:54.901379083 +0000 UTC m=+117.228943663"
Jul 24 15:48:54.916207 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.916115    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-bcnsq" podStartSLOduration=11.070674396 podStartE2EDuration="19.916089624s" totalImagesPullingTime="8.845415228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:54.914249589 +0000 UTC m=+117.241814140" watchObservedRunningTime="2026-07-24 15:48:54.916089624 +0000 UTC m=+117.243654156"
Jul 24 15:48:54.930403 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.930335    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vpr5g" podStartSLOduration=74.084845458 podStartE2EDuration="1m22.930311952s" totalImagesPullingTime="8.845466494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:54.929272797 +0000 UTC m=+117.256837334" watchObservedRunningTime="2026-07-24 15:48:54.930311952 +0000 UTC m=+117.257876503"
Jul 24 15:48:54.945214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:54.945154    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bcf75f55-bcsvp" podStartSLOduration=9.801290909 podStartE2EDuration="18.945136082s" totalImagesPullingTime="9.143845173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:54.943893434 +0000 UTC m=+117.271457972" watchObservedRunningTime="2026-07-24 15:48:54.945136082 +0000 UTC m=+117.272700620"
Jul 24 15:48:56.667381 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.667277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:56.893250 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.893195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:56.918312 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.918244    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"]
Jul 24 15:48:56.954917 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.954834    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rhxcd"]
Jul 24 15:48:56.955325 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.955056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:56.958356 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.958321    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 24 15:48:56.958356 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.958343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-w5dn5\""
Jul 24 15:48:56.958356 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.958350    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 24 15:48:56.978157 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.978102    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"]
Jul 24 15:48:56.978425 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.978181    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"]
Jul 24 15:48:56.978425 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.978284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:56.980697 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.980607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 15:48:56.980881 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.980751    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twmxq\""
Jul 24 15:48:56.981263 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.981144    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 15:48:56.981263 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.981217    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 15:48:56.983528 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.983497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:56.996897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.996864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vzkd\" (UniqueName: \"kubernetes.io/projected/b76a2700-72f3-49f6-b4b6-3c676a126e72-kube-api-access-2vzkd\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:56.997092 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.996986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:56.997183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.997089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:56.997183 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.997125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b76a2700-72f3-49f6-b4b6-3c676a126e72-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:56.998398 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.998362    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"]
Jul 24 15:48:56.998506 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:56.998484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.000790 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.000759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 15:48:57.001133 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.001079    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 15:48:57.001407 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.001280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ng2pt\""
Jul 24 15:48:57.001512 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.001487    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 15:48:57.059303 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.059267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.097755 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/690f9ba2-c7a6-464a-a550-81526d637617-metrics-client-ca\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.097755 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-tls\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.098022 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:57.097955    2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.097960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b76a2700-72f3-49f6-b4b6-3c676a126e72-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:48:57.098066    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-tls podName:b76a2700-72f3-49f6-b4b6-3c676a126e72 nodeName:}" failed. No retries permitted until 2026-07-24 15:48:57.598042808 +0000 UTC m=+119.925607338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-54tdb" (UID: "b76a2700-72f3-49f6-b4b6-3c676a126e72") : secret "openshift-state-metrics-tls" not found
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-accelerators-collector-config\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2nh9\" (UniqueName: \"kubernetes.io/projected/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-api-access-z2nh9\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-sys\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-wtmp\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vzkd\" (UniqueName: \"kubernetes.io/projected/b76a2700-72f3-49f6-b4b6-3c676a126e72-kube-api-access-2vzkd\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.098371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-textfile\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098889 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prtw2\" (UniqueName: \"kubernetes.io/projected/690f9ba2-c7a6-464a-a550-81526d637617-kube-api-access-prtw2\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098889 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-root\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.098889 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.098792    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b76a2700-72f3-49f6-b4b6-3c676a126e72-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.101199 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.101164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.106670 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.106600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vzkd\" (UniqueName: \"kubernetes.io/projected/b76a2700-72f3-49f6-b4b6-3c676a126e72-kube-api-access-2vzkd\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.160877 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.160840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.200175 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.199914    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:57.200175 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:57.200631 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-accelerators-collector-config\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.200763 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.200763 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2nh9\" (UniqueName: \"kubernetes.io/projected/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-api-access-z2nh9\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.200868 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.200868 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-sys\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.200989 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-wtmp\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.200989 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.200924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-textfile\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prtw2\" (UniqueName: \"kubernetes.io/projected/690f9ba2-c7a6-464a-a550-81526d637617-kube-api-access-prtw2\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-root\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/690f9ba2-c7a6-464a-a550-81526d637617-metrics-client-ca\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201243 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201243 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.201243 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-tls\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201396 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.201396 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.201396 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-wtmp\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201547 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-accelerators-collector-config\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.201751 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.201722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.202921 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.202593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-textfile\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.203075 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.203028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.203186 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.203155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.203266 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.203242    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-sys\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.203367 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.203340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/690f9ba2-c7a6-464a-a550-81526d637617-root\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.204071 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.204037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/690f9ba2-c7a6-464a-a550-81526d637617-metrics-client-ca\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.210995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.210826    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:57.216315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.216274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.218652 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.218602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prtw2\" (UniqueName: \"kubernetes.io/projected/690f9ba2-c7a6-464a-a550-81526d637617-kube-api-access-prtw2\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.219758 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.219186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-tls\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.220414 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.220337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/690f9ba2-c7a6-464a-a550-81526d637617-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rhxcd\" (UID: \"690f9ba2-c7a6-464a-a550-81526d637617\") " pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.222013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.221985    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2nh9\" (UniqueName: \"kubernetes.io/projected/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-api-access-z2nh9\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.222146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.222096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-bt4wc\" (UID: \"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.235931 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.235874    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.291905 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.291872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rhxcd"
Jul 24 15:48:57.292817 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.292778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.309841 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.309784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"
Jul 24 15:48:57.392146 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.392107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.492503 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.492423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.592961 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.592927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.604444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.604415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.607065 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.607044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b76a2700-72f3-49f6-b4b6-3c676a126e72-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-54tdb\" (UID: \"b76a2700-72f3-49f6-b4b6-3c676a126e72\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.871919 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.871884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"
Jul 24 15:48:57.895721 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.895684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.910966 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:57.910940    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:48:58.003543 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:58.003481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:58.130343 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:58.130266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:58.770491 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:58.770453    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690f9ba2_c7a6_464a_a550_81526d637617.slice/crio-e25e455f1f3f21f2c7a1a97d5f1bbd672b22991c3d879d7ee855d53d2bec75e7 WatchSource:0}: Error finding container e25e455f1f3f21f2c7a1a97d5f1bbd672b22991c3d879d7ee855d53d2bec75e7: Status 404 returned error can't find the container with id e25e455f1f3f21f2c7a1a97d5f1bbd672b22991c3d879d7ee855d53d2bec75e7
Jul 24 15:48:58.912050 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:58.911994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhxcd" event={"ID":"690f9ba2-c7a6-464a-a550-81526d637617","Type":"ContainerStarted","Data":"e25e455f1f3f21f2c7a1a97d5f1bbd672b22991c3d879d7ee855d53d2bec75e7"}
Jul 24 15:48:58.922453 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:58.922426    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc"]
Jul 24 15:48:58.924833 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:58.924806    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca9eae0_0a2b_4f7c_8c9c_2f1271bd6d91.slice/crio-832abd1ae6453a7bb9536a06ffaeac74e80568c5e8d53d02bb95f13753650b5a WatchSource:0}: Error finding container 832abd1ae6453a7bb9536a06ffaeac74e80568c5e8d53d02bb95f13753650b5a: Status 404 returned error can't find the container with id 832abd1ae6453a7bb9536a06ffaeac74e80568c5e8d53d02bb95f13753650b5a
Jul 24 15:48:58.939364 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:58.939248    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb"]
Jul 24 15:48:58.941760 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:48:58.941737    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76a2700_72f3_49f6_b4b6_3c676a126e72.slice/crio-a33773dc2d01f418908b90b1bef456a980d59f9c6e927ff251e75ba2bfd06822 WatchSource:0}: Error finding container a33773dc2d01f418908b90b1bef456a980d59f9c6e927ff251e75ba2bfd06822: Status 404 returned error can't find the container with id a33773dc2d01f418908b90b1bef456a980d59f9c6e927ff251e75ba2bfd06822
Jul 24 15:48:59.880721 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.880654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:59.918006 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.917863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb" event={"ID":"b76a2700-72f3-49f6-b4b6-3c676a126e72","Type":"ContainerStarted","Data":"1d11a12093d48a8de22c80d4ac38921655e4ec9438ab45ad5ef470fac0977855"}
Jul 24 15:48:59.918006 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.917939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb" event={"ID":"b76a2700-72f3-49f6-b4b6-3c676a126e72","Type":"ContainerStarted","Data":"ddc2c831b22f362a4088d81b728bf55a06a0ad2bbc4eb00047a3ba0a045b4fba"}
Jul 24 15:48:59.918006 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.917970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb" event={"ID":"b76a2700-72f3-49f6-b4b6-3c676a126e72","Type":"ContainerStarted","Data":"a33773dc2d01f418908b90b1bef456a980d59f9c6e927ff251e75ba2bfd06822"}
Jul 24 15:48:59.919218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.919140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc" event={"ID":"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91","Type":"ContainerStarted","Data":"832abd1ae6453a7bb9536a06ffaeac74e80568c5e8d53d02bb95f13753650b5a"}
Jul 24 15:48:59.921584 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.921558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pct5v" event={"ID":"11be8611-517c-405e-a8a3-4012c59d082a","Type":"ContainerStarted","Data":"edb969ccaf5b703ffbc4f4f0138964db5e0f695b753b3816cd167819df7ec2ab"}
Jul 24 15:48:59.941546 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.941488    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pct5v" podStartSLOduration=33.922434883 podStartE2EDuration="59.941468944s" totalImagesPullingTime="26.019034061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:48:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:59.940191755 +0000 UTC m=+122.267756319" watchObservedRunningTime="2026-07-24 15:48:59.941468944 +0000 UTC m=+122.269033484"
Jul 24 15:48:59.942901 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:48:59.942869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:00.028498 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:00.028460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:00.129492 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:00.129458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:00.927043 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:00.926965    2577 generic.go:356] "Generic (PLEG): container finished" podID="690f9ba2-c7a6-464a-a550-81526d637617" containerID="eb3008ab44af8612872603483585ed59d587acb90812aa56d14d1eec79a84251" exitCode=0
Jul 24 15:49:00.927584 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:00.927051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhxcd" event={"ID":"690f9ba2-c7a6-464a-a550-81526d637617","Type":"ContainerDied","Data":"eb3008ab44af8612872603483585ed59d587acb90812aa56d14d1eec79a84251"}
Jul 24 15:49:01.136570 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.136529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:01.344163 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.344128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:01.728347 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.728303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:01.747681 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.747652    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8d6694c4d-zw9qt"]
Jul 24 15:49:01.753421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.753345    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"]
Jul 24 15:49:01.773468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.773424    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"]
Jul 24 15:49:01.773708 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.773635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.776277 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.776242    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 15:49:01.776885 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.776862    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-q8wng\""
Jul 24 15:49:01.799538 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.799506    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5858f9874b-q2995"]
Jul 24 15:49:01.815788 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.815750    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5858f9874b-q2995"]
Jul 24 15:49:01.815933 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.815868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.825531 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.825499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:01.847282 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.847233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsg28\" (UniqueName: \"kubernetes.io/projected/1bf748a1-0011-4055-ad7c-e9b8c8a8deb3-kube-api-access-dsg28\") pod \"monitoring-plugin-68b7fc9574-6ckzh\" (UID: \"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.847505 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.847312    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1bf748a1-0011-4055-ad7c-e9b8c8a8deb3-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-6ckzh\" (UID: \"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.926622 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.926574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:01.933060 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.933032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhxcd" event={"ID":"690f9ba2-c7a6-464a-a550-81526d637617","Type":"ContainerStarted","Data":"6b1d96358035f42caffae2d5b66aba5b70fd859c07b86071f24daacebc6a5fb6"}
Jul 24 15:49:01.933432 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.933085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhxcd" event={"ID":"690f9ba2-c7a6-464a-a550-81526d637617","Type":"ContainerStarted","Data":"dbc08525253b1272aed9c46378ef0d4e11b2b5ab69b0913e43c1db645e846273"}
Jul 24 15:49:01.935221 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.935194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc" event={"ID":"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91","Type":"ContainerStarted","Data":"aa4ef8a5f2d70496b8a286ea7805dbd47f73b4a6f3b0d9857d3c295ca6d4dd69"}
Jul 24 15:49:01.935351 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.935244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc" event={"ID":"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91","Type":"ContainerStarted","Data":"5c0837a6ae5961f2b78aea4482e23bce166e896b52c8e6fd82a80e5daad4adc2"}
Jul 24 15:49:01.935351 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.935273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc" event={"ID":"cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91","Type":"ContainerStarted","Data":"0f7c13f66418f69590eae02b28d31f1ffa88571ee0bf412bcea52d014b9a5361"}
Jul 24 15:49:01.937255 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.937233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb" event={"ID":"b76a2700-72f3-49f6-b4b6-3c676a126e72","Type":"ContainerStarted","Data":"18f7d832bc6c02ff3864a9bf8793e36aba3b4deed27ae5f4b744bb81c01b1b9d"}
Jul 24 15:49:01.948830 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.948800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-oauth-config\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.948945 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.948883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-oauth-serving-cert\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.948945 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.948914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-trusted-ca-bundle\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.949059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.948953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsg28\" (UniqueName: \"kubernetes.io/projected/1bf748a1-0011-4055-ad7c-e9b8c8a8deb3-kube-api-access-dsg28\") pod \"monitoring-plugin-68b7fc9574-6ckzh\" (UID: \"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.949059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.949021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdwf5\" (UniqueName: \"kubernetes.io/projected/74dd407a-9a46-4c23-bf55-5b0f3e01997d-kube-api-access-zdwf5\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.949059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.949039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1bf748a1-0011-4055-ad7c-e9b8c8a8deb3-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-6ckzh\" (UID: \"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.949212 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.949068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-serving-cert\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.949212 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.949092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-config\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.949361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.949317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-service-ca\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:01.950384 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.950339    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rhxcd" podStartSLOduration=4.52155024 podStartE2EDuration="5.950321166s" totalImagesPullingTime="1.32150189s" totalInitContainerRuntime="107.269036ms" isStatefulPod=true podCreationTimestamp="2026-07-24 15:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:01.948900149 +0000 UTC m=+124.276464682" watchObservedRunningTime="2026-07-24 15:49:01.950321166 +0000 UTC m=+124.277885756"
Jul 24 15:49:01.951757 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.951737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1bf748a1-0011-4055-ad7c-e9b8c8a8deb3-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-6ckzh\" (UID: \"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.956378 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.956357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsg28\" (UniqueName: \"kubernetes.io/projected/1bf748a1-0011-4055-ad7c-e9b8c8a8deb3-kube-api-access-dsg28\") pod \"monitoring-plugin-68b7fc9574-6ckzh\" (UID: \"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:01.966395 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.966330    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-bt4wc" podStartSLOduration=3.581370653 podStartE2EDuration="5.96631171s" totalImagesPullingTime="2.384941057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:01.964647186 +0000 UTC m=+124.292211725" watchObservedRunningTime="2026-07-24 15:49:01.96631171 +0000 UTC m=+124.293876249"
Jul 24 15:49:01.986249 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:01.986140    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-54tdb" podStartSLOduration=4.038261221 podStartE2EDuration="5.98610411s" totalImagesPullingTime="1.947842889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:01.984183431 +0000 UTC m=+124.311747970" watchObservedRunningTime="2026-07-24 15:49:01.98610411 +0000 UTC m=+124.313668652"
Jul 24 15:49:02.050759 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.050674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdwf5\" (UniqueName: \"kubernetes.io/projected/74dd407a-9a46-4c23-bf55-5b0f3e01997d-kube-api-access-zdwf5\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.050759 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.050741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-serving-cert\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051051 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.050769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-config\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051051 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.050939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-service-ca\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051164 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.051088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-oauth-config\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051220 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.051209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-oauth-serving-cert\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051275 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.051249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-trusted-ca-bundle\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051894 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.051527    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-config\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.051894 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.051665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-service-ca\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.052069 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.051924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-oauth-serving-cert\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.052261 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.052238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-trusted-ca-bundle\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.053576 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.053559    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-oauth-config\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.053782 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.053766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-serving-cert\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.057885 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.057868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdwf5\" (UniqueName: \"kubernetes.io/projected/74dd407a-9a46-4c23-bf55-5b0f3e01997d-kube-api-access-zdwf5\") pod \"console-5858f9874b-q2995\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") " pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.090105 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.090046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:02.125672 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.125606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:02.266120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.266094    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"]
Jul 24 15:49:02.268776 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:49:02.268741    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf748a1_0011_4055_ad7c_e9b8c8a8deb3.slice/crio-e02079b125281696e2212899b9d9814f37163f7ab96339a3ca1268ec0683cbb1 WatchSource:0}: Error finding container e02079b125281696e2212899b9d9814f37163f7ab96339a3ca1268ec0683cbb1: Status 404 returned error can't find the container with id e02079b125281696e2212899b9d9814f37163f7ab96339a3ca1268ec0683cbb1
Jul 24 15:49:02.299032 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.298959    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5858f9874b-q2995"]
Jul 24 15:49:02.305221 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:49:02.305173    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74dd407a_9a46_4c23_bf55_5b0f3e01997d.slice/crio-6dfc3d997219d7ee18a0999eb782ed1b258ec3f8f829c284f6fc286c36ab053a WatchSource:0}: Error finding container 6dfc3d997219d7ee18a0999eb782ed1b258ec3f8f829c284f6fc286c36ab053a: Status 404 returned error can't find the container with id 6dfc3d997219d7ee18a0999eb782ed1b258ec3f8f829c284f6fc286c36ab053a
Jul 24 15:49:02.510059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.510020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:02.610332 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.610243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:02.948605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.948147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858f9874b-q2995" event={"ID":"74dd407a-9a46-4c23-bf55-5b0f3e01997d","Type":"ContainerStarted","Data":"113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352"}
Jul 24 15:49:02.948605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.948254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858f9874b-q2995" event={"ID":"74dd407a-9a46-4c23-bf55-5b0f3e01997d","Type":"ContainerStarted","Data":"6dfc3d997219d7ee18a0999eb782ed1b258ec3f8f829c284f6fc286c36ab053a"}
Jul 24 15:49:02.949694 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.949667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh" event={"ID":"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3","Type":"ContainerStarted","Data":"e02079b125281696e2212899b9d9814f37163f7ab96339a3ca1268ec0683cbb1"}
Jul 24 15:49:02.967543 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:02.967462    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5858f9874b-q2995" podStartSLOduration=1.967430581 podStartE2EDuration="1.967430581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:02.964411084 +0000 UTC m=+125.291975623" watchObservedRunningTime="2026-07-24 15:49:02.967430581 +0000 UTC m=+125.294995120"
Jul 24 15:49:03.194892 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:03.194858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:03.945896 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:03.945864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:49:04.900994 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:04.900968    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vpr5g"
Jul 24 15:49:04.962199 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:04.962136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh" event={"ID":"1bf748a1-0011-4055-ad7c-e9b8c8a8deb3","Type":"ContainerStarted","Data":"1b1e95f06b762f999cb0d2c63fdc98b4675f7695d5f419e6ad0e7d4574f2f489"}
Jul 24 15:49:04.962555 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:04.962376    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:04.968710 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:04.968678    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh"
Jul 24 15:49:04.980685 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:04.980594    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-6ckzh" podStartSLOduration=2.224720227 podStartE2EDuration="3.980575194s" totalImagesPullingTime="1.755854967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:04.980132476 +0000 UTC m=+127.307697014" watchObservedRunningTime="2026-07-24 15:49:04.980575194 +0000 UTC m=+127.308139812"
Jul 24 15:49:06.124293 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.124259    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5858f9874b-q2995"]
Jul 24 15:49:06.157546 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.157509    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5864667bff-xqw8g"]
Jul 24 15:49:06.177995 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.177961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5864667bff-xqw8g"]
Jul 24 15:49:06.178156 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.178059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297115 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-trusted-ca-bundle\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297456 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-oauth-serving-cert\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297456 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wm5p\" (UniqueName: \"kubernetes.io/projected/57fe378f-7b4e-46be-a231-17f17d4c9250-kube-api-access-7wm5p\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297456 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-console-config\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297456 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-service-ca\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297456 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-serving-cert\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.297712 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.297492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-oauth-config\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399102 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-oauth-serving-cert\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399102 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wm5p\" (UniqueName: \"kubernetes.io/projected/57fe378f-7b4e-46be-a231-17f17d4c9250-kube-api-access-7wm5p\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-console-config\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-service-ca\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-serving-cert\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-oauth-config\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.399358 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.399287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-trusted-ca-bundle\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.400285 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.400224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-oauth-serving-cert\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.400285 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.400254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-service-ca\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.400566 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.400381    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-console-config\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.400942 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.400903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-trusted-ca-bundle\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.403765 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.403734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-serving-cert\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.403765 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.403755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-oauth-config\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.411551 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.411505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wm5p\" (UniqueName: \"kubernetes.io/projected/57fe378f-7b4e-46be-a231-17f17d4c9250-kube-api-access-7wm5p\") pod \"console-5864667bff-xqw8g\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") " pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.492664 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.492602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:06.690998 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.690925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5864667bff-xqw8g"]
Jul 24 15:49:06.694485 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:49:06.694453    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57fe378f_7b4e_46be_a231_17f17d4c9250.slice/crio-28722ee3ebe932b364d4f78078f0db2859a81a096566f5f9f991383bd659c5fc WatchSource:0}: Error finding container 28722ee3ebe932b364d4f78078f0db2859a81a096566f5f9f991383bd659c5fc: Status 404 returned error can't find the container with id 28722ee3ebe932b364d4f78078f0db2859a81a096566f5f9f991383bd659c5fc
Jul 24 15:49:06.974323 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.974198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5864667bff-xqw8g" event={"ID":"57fe378f-7b4e-46be-a231-17f17d4c9250","Type":"ContainerStarted","Data":"64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c"}
Jul 24 15:49:06.974323 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.974269    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5864667bff-xqw8g" event={"ID":"57fe378f-7b4e-46be-a231-17f17d4c9250","Type":"ContainerStarted","Data":"28722ee3ebe932b364d4f78078f0db2859a81a096566f5f9f991383bd659c5fc"}
Jul 24 15:49:06.997049 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:06.996988    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5864667bff-xqw8g" podStartSLOduration=0.996970774 podStartE2EDuration="996.970774ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:06.995206089 +0000 UTC m=+129.322770631" watchObservedRunningTime="2026-07-24 15:49:06.996970774 +0000 UTC m=+129.324535312"
Jul 24 15:49:07.004707 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:07.004677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:49:07.006955 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:07.006937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a854e3ef-7cf7-467e-896b-339ed5f3acc6-metrics-certs\") pod \"network-metrics-daemon-pc2tq\" (UID: \"a854e3ef-7cf7-467e-896b-339ed5f3acc6\") " pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:49:07.148589 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:07.148559    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k45mm\""
Jul 24 15:49:07.157285 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:07.157260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pc2tq"
Jul 24 15:49:07.306362 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:07.306328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pc2tq"]
Jul 24 15:49:07.310145 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:49:07.310110    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda854e3ef_7cf7_467e_896b_339ed5f3acc6.slice/crio-8c1566dd9f94c98176be0c604d801a5d67638b50b139e4391f1b0a6c2cbf79a1 WatchSource:0}: Error finding container 8c1566dd9f94c98176be0c604d801a5d67638b50b139e4391f1b0a6c2cbf79a1: Status 404 returned error can't find the container with id 8c1566dd9f94c98176be0c604d801a5d67638b50b139e4391f1b0a6c2cbf79a1
Jul 24 15:49:07.983382 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:07.983341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pc2tq" event={"ID":"a854e3ef-7cf7-467e-896b-339ed5f3acc6","Type":"ContainerStarted","Data":"8c1566dd9f94c98176be0c604d801a5d67638b50b139e4391f1b0a6c2cbf79a1"}
Jul 24 15:49:08.863148 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:08.863096    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:49:09.994196 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:09.994164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pc2tq" event={"ID":"a854e3ef-7cf7-467e-896b-339ed5f3acc6","Type":"ContainerStarted","Data":"0e6bf9a9944885a26d3dd807657614df9fc6813b9e825455b4c51d6da04dec9c"}
Jul 24 15:49:09.994673 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:09.994212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pc2tq" event={"ID":"a854e3ef-7cf7-467e-896b-339ed5f3acc6","Type":"ContainerStarted","Data":"d60846525e0abb0c92e6a58fdaafe21e99c54c632c822831839cace8a2b7c362"}
Jul 24 15:49:10.011695 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:10.011647    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pc2tq" podStartSLOduration=130.33149487 podStartE2EDuration="2m12.011606154s" totalImagesPullingTime="1.680111284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:10.009375831 +0000 UTC m=+132.336940369" watchObservedRunningTime="2026-07-24 15:49:10.011606154 +0000 UTC m=+132.339170691"
Jul 24 15:49:10.102434 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:10.102402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:10.163145 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:10.163098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:12.126420 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:12.126387    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:12.694359 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:12.694324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:12.746561 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:12.746507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:13.895689 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:13.895602    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6" podUID="f060f90a-0b0c-48d5-8199-5edf77517497" containerName="registry" containerID="cri-o://46a2dc00cc99a7760ec59d5c4b7d27842dbb912d14384d55ef2760bf7dc1034c" gracePeriod=30
Jul 24 15:49:14.013970 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.013928    2577 generic.go:356] "Generic (PLEG): container finished" podID="f060f90a-0b0c-48d5-8199-5edf77517497" containerID="46a2dc00cc99a7760ec59d5c4b7d27842dbb912d14384d55ef2760bf7dc1034c" exitCode=0
Jul 24 15:49:14.014116 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.014004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6" event={"ID":"f060f90a-0b0c-48d5-8199-5edf77517497","Type":"ContainerDied","Data":"46a2dc00cc99a7760ec59d5c4b7d27842dbb912d14384d55ef2760bf7dc1034c"}
Jul 24 15:49:14.022548 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.022504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:14.074277 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.074239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:14.209817 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.209785    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:49:14.381457 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381380    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-installation-pull-secrets\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381457 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381470    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-bound-sa-token\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381558    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-image-registry-private-configuration\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381589    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-trusted-ca\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381728    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwb82\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-kube-api-access-zwb82\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f060f90a-0b0c-48d5-8199-5edf77517497-ca-trust-extracted\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.381951 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.381893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-registry-certificates\") pod \"f060f90a-0b0c-48d5-8199-5edf77517497\" (UID: \"f060f90a-0b0c-48d5-8199-5edf77517497\") "
Jul 24 15:49:14.382812 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.382655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:14.382812 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.382741    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:14.385513 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.385454    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:14.385513 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.385483    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:14.385774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.385454    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:14.385774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.385585    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-kube-api-access-zwb82" (OuterVolumeSpecName: "kube-api-access-zwb82") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "kube-api-access-zwb82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:14.385774 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.385700    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:14.392366 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.392288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f060f90a-0b0c-48d5-8199-5edf77517497-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f060f90a-0b0c-48d5-8199-5edf77517497" (UID: "f060f90a-0b0c-48d5-8199-5edf77517497"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483246    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-image-registry-private-configuration\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483308    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-trusted-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483323    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwb82\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-kube-api-access-zwb82\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483333    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f060f90a-0b0c-48d5-8199-5edf77517497-ca-trust-extracted\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483344    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f060f90a-0b0c-48d5-8199-5edf77517497-registry-certificates\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483353    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f060f90a-0b0c-48d5-8199-5edf77517497-installation-pull-secrets\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483365    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-bound-sa-token\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.483417 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:14.483375    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f060f90a-0b0c-48d5-8199-5edf77517497-registry-tls\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:15.019882 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:15.019849    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6"
Jul 24 15:49:15.020290 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:15.019852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68dbdf489c-5rvb6" event={"ID":"f060f90a-0b0c-48d5-8199-5edf77517497","Type":"ContainerDied","Data":"58e63e89b879d12345f2e648be0045fe7ce147f63c21d1b8482e491073677717"}
Jul 24 15:49:15.020290 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:15.019992    2577 scope.go:117] "RemoveContainer" containerID="46a2dc00cc99a7760ec59d5c4b7d27842dbb912d14384d55ef2760bf7dc1034c"
Jul 24 15:49:15.041849 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:15.041764    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68dbdf489c-5rvb6"]
Jul 24 15:49:15.044481 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:15.044447    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68dbdf489c-5rvb6"]
Jul 24 15:49:16.239912 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:16.239841    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f060f90a-0b0c-48d5-8199-5edf77517497" path="/var/lib/kubelet/pods/f060f90a-0b0c-48d5-8199-5edf77517497/volumes"
Jul 24 15:49:16.493090 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:16.492979    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:16.493090 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:16.493062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:16.498198 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:16.498167    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:17.039008 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:17.038976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:49:17.084673 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:17.084630    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bcf75f55-bcsvp"]
Jul 24 15:49:17.203268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:17.203234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:17.301413 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:17.301326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:17.400339 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:17.400299    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:23.733927 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:23.733883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:23.795976 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:23.795943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:23.889481 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:23.889422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:26.772023 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:26.771987    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8d6694c4d-zw9qt" podUID="a675627a-3c12-46c5-a633-b22e9afffbd3" containerName="console" containerID="cri-o://da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4" gracePeriod=15
Jul 24 15:49:27.054692 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.054635    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:49:27.076174 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.076140    2577 generic.go:356] "Generic (PLEG): container finished" podID="a675627a-3c12-46c5-a633-b22e9afffbd3" containerID="da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4" exitCode=0
Jul 24 15:49:27.076361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.076189    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d6694c4d-zw9qt"
Jul 24 15:49:27.076361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.076215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d6694c4d-zw9qt" event={"ID":"a675627a-3c12-46c5-a633-b22e9afffbd3","Type":"ContainerDied","Data":"da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4"}
Jul 24 15:49:27.076361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.076271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d6694c4d-zw9qt" event={"ID":"a675627a-3c12-46c5-a633-b22e9afffbd3","Type":"ContainerDied","Data":"a5b9a0d5fd268cbfc3376aec3f5562e8ab5bf84c691312f17e17568977be3d65"}
Jul 24 15:49:27.076361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.076299    2577 scope.go:117] "RemoveContainer" containerID="da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4"
Jul 24 15:49:27.084268 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.084253    2577 scope.go:117] "RemoveContainer" containerID="da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4"
Jul 24 15:49:27.084560 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:49:27.084541    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4\": container with ID starting with da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4 not found: ID does not exist" containerID="da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4"
Jul 24 15:49:27.084607 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.084568    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4"} err="failed to get container status \"da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4\": rpc error: code = NotFound desc = could not find container \"da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4\": container with ID starting with da85b38cc11cf8bd167fba40f731149897373caa72d4ccbfe2ca75a75c8a89f4 not found: ID does not exist"
Jul 24 15:49:27.094988 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.094968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-oauth-serving-cert\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095093 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-oauth-config\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095093 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-service-ca\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095093 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095087    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-serving-cert\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-trusted-ca-bundle\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75sv5\" (UniqueName: \"kubernetes.io/projected/a675627a-3c12-46c5-a633-b22e9afffbd3-kube-api-access-75sv5\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-console-config\") pod \"a675627a-3c12-46c5-a633-b22e9afffbd3\" (UID: \"a675627a-3c12-46c5-a633-b22e9afffbd3\") "
Jul 24 15:49:27.095464 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:27.095464 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095439    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-service-ca" (OuterVolumeSpecName: "service-ca") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:27.095632 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095495    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:27.095755 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.095737    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-console-config" (OuterVolumeSpecName: "console-config") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:27.097170 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.097150    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:27.097601 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.097576    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a675627a-3c12-46c5-a633-b22e9afffbd3-kube-api-access-75sv5" (OuterVolumeSpecName: "kube-api-access-75sv5") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "kube-api-access-75sv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:27.097601 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.097584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a675627a-3c12-46c5-a633-b22e9afffbd3" (UID: "a675627a-3c12-46c5-a633-b22e9afffbd3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:27.196224 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196192    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.196224 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196219    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-trusted-ca-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.196224 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196228    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75sv5\" (UniqueName: \"kubernetes.io/projected/a675627a-3c12-46c5-a633-b22e9afffbd3-kube-api-access-75sv5\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.196435 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196237    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-console-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.196435 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196245    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-oauth-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.196435 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196253    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a675627a-3c12-46c5-a633-b22e9afffbd3-console-oauth-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.196435 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.196265    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a675627a-3c12-46c5-a633-b22e9afffbd3-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:27.395862 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.395833    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8d6694c4d-zw9qt"]
Jul 24 15:49:27.400905 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.400881    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8d6694c4d-zw9qt"]
Jul 24 15:49:27.699974 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.699889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:27.750592 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:27.750561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:28.237921 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:28.237890    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a675627a-3c12-46c5-a633-b22e9afffbd3" path="/var/lib/kubelet/pods/a675627a-3c12-46c5-a633-b22e9afffbd3/volumes"
Jul 24 15:49:31.105192 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.105162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:31.148251 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.148221    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5858f9874b-q2995" podUID="74dd407a-9a46-4c23-bf55-5b0f3e01997d" containerName="console" containerID="cri-o://113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352" gracePeriod=15
Jul 24 15:49:31.156483 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.156460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:31.424446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.424422    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:31.529180 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529149    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdwf5\" (UniqueName: \"kubernetes.io/projected/74dd407a-9a46-4c23-bf55-5b0f3e01997d-kube-api-access-zdwf5\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529319 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529189    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-service-ca\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529319 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529230    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-config\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529319 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529259    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-serving-cert\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529319 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529288    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-trusted-ca-bundle\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529632 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-oauth-config\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529724 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529606    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-service-ca" (OuterVolumeSpecName: "service-ca") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:31.529724 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529629    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-config" (OuterVolumeSpecName: "console-config") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:31.529724 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529668    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-oauth-serving-cert\") pod \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\" (UID: \"74dd407a-9a46-4c23-bf55-5b0f3e01997d\") "
Jul 24 15:49:31.529873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529717    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:31.529921 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529895    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:31.529921 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529913    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:31.530000 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.529948    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-trusted-ca-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:31.530100 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.530057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:31.531515 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.531490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:31.531827 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.531809    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:31.531904 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.531838    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74dd407a-9a46-4c23-bf55-5b0f3e01997d-kube-api-access-zdwf5" (OuterVolumeSpecName: "kube-api-access-zdwf5") pod "74dd407a-9a46-4c23-bf55-5b0f3e01997d" (UID: "74dd407a-9a46-4c23-bf55-5b0f3e01997d"). InnerVolumeSpecName "kube-api-access-zdwf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:31.630420 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.630398    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdwf5\" (UniqueName: \"kubernetes.io/projected/74dd407a-9a46-4c23-bf55-5b0f3e01997d-kube-api-access-zdwf5\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:31.630420 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.630418    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:31.630542 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.630427    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74dd407a-9a46-4c23-bf55-5b0f3e01997d-console-oauth-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:31.630542 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:31.630436    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74dd407a-9a46-4c23-bf55-5b0f3e01997d-oauth-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:32.096751 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.096712    2577 generic.go:356] "Generic (PLEG): container finished" podID="74dd407a-9a46-4c23-bf55-5b0f3e01997d" containerID="113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352" exitCode=0
Jul 24 15:49:32.096919 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.096761    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858f9874b-q2995"
Jul 24 15:49:32.096919 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.096794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858f9874b-q2995" event={"ID":"74dd407a-9a46-4c23-bf55-5b0f3e01997d","Type":"ContainerDied","Data":"113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352"}
Jul 24 15:49:32.096919 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.096845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858f9874b-q2995" event={"ID":"74dd407a-9a46-4c23-bf55-5b0f3e01997d","Type":"ContainerDied","Data":"6dfc3d997219d7ee18a0999eb782ed1b258ec3f8f829c284f6fc286c36ab053a"}
Jul 24 15:49:32.096919 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.096865    2577 scope.go:117] "RemoveContainer" containerID="113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352"
Jul 24 15:49:32.105142 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.105125    2577 scope.go:117] "RemoveContainer" containerID="113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352"
Jul 24 15:49:32.105374 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:49:32.105357    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352\": container with ID starting with 113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352 not found: ID does not exist" containerID="113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352"
Jul 24 15:49:32.105633 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.105382    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352"} err="failed to get container status \"113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352\": rpc error: code = NotFound desc = could not find container \"113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352\": container with ID starting with 113414852a64f115df9e41ce26122cfe0c9931f68d90eeaea68a819174bb7352 not found: ID does not exist"
Jul 24 15:49:32.116050 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.116020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5858f9874b-q2995"]
Jul 24 15:49:32.121985 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.121901    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5858f9874b-q2995"]
Jul 24 15:49:32.237194 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:32.237165    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74dd407a-9a46-4c23-bf55-5b0f3e01997d" path="/var/lib/kubelet/pods/74dd407a-9a46-4c23-bf55-5b0f3e01997d/volumes"
Jul 24 15:49:34.713541 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:34.713504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:34.814873 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:34.814845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:34.866189 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:34.866160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:36.114801 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:36.114763    2577 generic.go:356] "Generic (PLEG): container finished" podID="70f17e22-92d6-46f7-b95f-1ffc1fd2731b" containerID="8a750ca28e55548c52403f685f4ae09c3f587462ecabcacea9289b52d8e2b64a" exitCode=0
Jul 24 15:49:36.115154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:36.114832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-msjdg" event={"ID":"70f17e22-92d6-46f7-b95f-1ffc1fd2731b","Type":"ContainerDied","Data":"8a750ca28e55548c52403f685f4ae09c3f587462ecabcacea9289b52d8e2b64a"}
Jul 24 15:49:36.115154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:36.115136    2577 scope.go:117] "RemoveContainer" containerID="8a750ca28e55548c52403f685f4ae09c3f587462ecabcacea9289b52d8e2b64a"
Jul 24 15:49:37.120224 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:37.120195    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-msjdg" event={"ID":"70f17e22-92d6-46f7-b95f-1ffc1fd2731b","Type":"ContainerStarted","Data":"ca7cacae533f63fb2d050c7cf9ba5bfb24999d11ac8ad221f7925a9cc2e0c066"}
Jul 24 15:49:37.140944 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:37.140869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:37.204442 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:37.204410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:42.109656 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.109607    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bcf75f55-bcsvp" podUID="b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" containerName="console" containerID="cri-o://6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3" gracePeriod=15
Jul 24 15:49:42.368807 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.368746    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:49:42.407592 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407551    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-config\") pod \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") "
Jul 24 15:49:42.407789 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-service-ca\") pod \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") "
Jul 24 15:49:42.407789 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407685    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg9wf\" (UniqueName: \"kubernetes.io/projected/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-kube-api-access-pg9wf\") pod \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") "
Jul 24 15:49:42.407789 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-serving-cert\") pod \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") "
Jul 24 15:49:42.407789 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407747    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-oauth-serving-cert\") pod \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") "
Jul 24 15:49:42.408005 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-oauth-config\") pod \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\" (UID: \"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9\") "
Jul 24 15:49:42.408060 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.407986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-config" (OuterVolumeSpecName: "console-config") pod "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" (UID: "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:42.415507 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.408922    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-service-ca" (OuterVolumeSpecName: "service-ca") pod "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" (UID: "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:42.415507 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.408969    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" (UID: "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:49:42.415507 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.409640    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:42.415507 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.409709    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:42.415507 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.409723    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-oauth-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:42.415507 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.410584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-kube-api-access-pg9wf" (OuterVolumeSpecName: "kube-api-access-pg9wf") pod "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" (UID: "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9"). InnerVolumeSpecName "kube-api-access-pg9wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:42.417278 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.415918    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" (UID: "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:42.417278 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.416232    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" (UID: "b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:42.511128 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.511094    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg9wf\" (UniqueName: \"kubernetes.io/projected/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-kube-api-access-pg9wf\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:42.511128 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.511124    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:42.511128 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:42.511136    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9-console-oauth-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:49:43.143603 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.143564    2577 generic.go:356] "Generic (PLEG): container finished" podID="b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" containerID="6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3" exitCode=0
Jul 24 15:49:43.144051 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.143647    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bcf75f55-bcsvp"
Jul 24 15:49:43.144051 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.143644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bcf75f55-bcsvp" event={"ID":"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9","Type":"ContainerDied","Data":"6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3"}
Jul 24 15:49:43.144051 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.143717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bcf75f55-bcsvp" event={"ID":"b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9","Type":"ContainerDied","Data":"d8c43d623df7e1658c750bb1f69c1e8373257b12ba9c85fdcfc781b3fbc232d7"}
Jul 24 15:49:43.144051 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.143737    2577 scope.go:117] "RemoveContainer" containerID="6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3"
Jul 24 15:49:43.151962 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.151947    2577 scope.go:117] "RemoveContainer" containerID="6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3"
Jul 24 15:49:43.152208 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:49:43.152191    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3\": container with ID starting with 6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3 not found: ID does not exist" containerID="6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3"
Jul 24 15:49:43.152269 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.152215    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3"} err="failed to get container status \"6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3\": rpc error: code = NotFound desc = could not find container \"6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3\": container with ID starting with 6a683290f40e91c75f7201702e434e255bc3e3bcd1d4a7013017c8267903d1b3 not found: ID does not exist"
Jul 24 15:49:43.178273 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.178250    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bcf75f55-bcsvp"]
Jul 24 15:49:43.182006 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:43.181987    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bcf75f55-bcsvp"]
Jul 24 15:49:44.237103 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:49:44.237070    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" path="/var/lib/kubelet/pods/b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9/volumes"
Jul 24 15:50:13.059563 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:13.059528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:13.148085 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:13.148057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:21.320053 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320017    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c8884465d-wkxv8"]
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320284    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f060f90a-0b0c-48d5-8199-5edf77517497" containerName="registry"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320295    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f060f90a-0b0c-48d5-8199-5edf77517497" containerName="registry"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320306    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74dd407a-9a46-4c23-bf55-5b0f3e01997d" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320311    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="74dd407a-9a46-4c23-bf55-5b0f3e01997d" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320322    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320327    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320340    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a675627a-3c12-46c5-a633-b22e9afffbd3" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320345    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a675627a-3c12-46c5-a633-b22e9afffbd3" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320387    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d76a7a-c501-4c15-b7fb-0949bc6bf4f9" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320395    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="74dd407a-9a46-4c23-bf55-5b0f3e01997d" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320402    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a675627a-3c12-46c5-a633-b22e9afffbd3" containerName="console"
Jul 24 15:50:21.320421 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.320408    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f060f90a-0b0c-48d5-8199-5edf77517497" containerName="registry"
Jul 24 15:50:21.322225 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.322205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.331171 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.331151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8884465d-wkxv8"]
Jul 24 15:50:21.412280 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.412247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:21.415092 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-serving-cert\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.415170 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-config\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.415170 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-oauth-config\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.415247 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-service-ca\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.415247 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-trusted-ca-bundle\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.415308 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czfqd\" (UniqueName: \"kubernetes.io/projected/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-kube-api-access-czfqd\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.415341 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.415307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-oauth-serving-cert\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.467069 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.467035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:21.516796 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.516757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-oauth-serving-cert\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.517066 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.517049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-serving-cert\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.517234 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.517220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-config\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.517692 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.517666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-oauth-serving-cert\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.517843 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.517826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-oauth-config\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.518239 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.518224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-service-ca\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.518741 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.518715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-trusted-ca-bundle\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.518826 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.518771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czfqd\" (UniqueName: \"kubernetes.io/projected/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-kube-api-access-czfqd\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.518937 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.517980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-config\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.519191 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.519173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-service-ca\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.519808 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.519787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-trusted-ca-bundle\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.520869 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.520847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-serving-cert\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.520949 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.520915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-oauth-config\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.527189 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.527168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czfqd\" (UniqueName: \"kubernetes.io/projected/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-kube-api-access-czfqd\") pod \"console-7c8884465d-wkxv8\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") " pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.631785 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.631697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:21.772144 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:21.772111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8884465d-wkxv8"]
Jul 24 15:50:21.776371 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:50:21.776342    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b336f4c_9c24_42d6_8ef2_39fd2dd39d42.slice/crio-72282dd85a378dc77cec6a46d33be98b6d69fdda2a89776036004a62f12820cf WatchSource:0}: Error finding container 72282dd85a378dc77cec6a46d33be98b6d69fdda2a89776036004a62f12820cf: Status 404 returned error can't find the container with id 72282dd85a378dc77cec6a46d33be98b6d69fdda2a89776036004a62f12820cf
Jul 24 15:50:22.295145 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:22.295105    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8884465d-wkxv8" event={"ID":"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42","Type":"ContainerStarted","Data":"b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80"}
Jul 24 15:50:22.295312 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:22.295164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8884465d-wkxv8" event={"ID":"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42","Type":"ContainerStarted","Data":"72282dd85a378dc77cec6a46d33be98b6d69fdda2a89776036004a62f12820cf"}
Jul 24 15:50:22.311148 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:22.311095    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c8884465d-wkxv8" podStartSLOduration=1.311076984 podStartE2EDuration="1.311076984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:50:22.30980999 +0000 UTC m=+204.637374528" watchObservedRunningTime="2026-07-24 15:50:22.311076984 +0000 UTC m=+204.638641523"
Jul 24 15:50:25.050670 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:25.050630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:25.152162 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:25.152129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:27.762396 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:27.762360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:27.860595 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:27.860568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:31.632410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:31.632375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:31.632936 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:31.632440    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:31.637003 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:31.636984    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:32.337716 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:32.337687    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:50:32.382561 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:32.382525    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5864667bff-xqw8g"]
Jul 24 15:50:32.499544 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:32.499510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:32.600341 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:32.600239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:32.660046 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:32.660017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:57.402925 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.402867    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5864667bff-xqw8g" podUID="57fe378f-7b4e-46be-a231-17f17d4c9250" containerName="console" containerID="cri-o://64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c" gracePeriod=15
Jul 24 15:50:57.655112 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.655027    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:50:57.719787 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-oauth-serving-cert\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.719969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719796    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-oauth-config\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.719969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719849    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-trusted-ca-bundle\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.719969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wm5p\" (UniqueName: \"kubernetes.io/projected/57fe378f-7b4e-46be-a231-17f17d4c9250-kube-api-access-7wm5p\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.719969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-console-config\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.719969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719949    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-service-ca\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.720242 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.719990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-serving-cert\") pod \"57fe378f-7b4e-46be-a231-17f17d4c9250\" (UID: \"57fe378f-7b4e-46be-a231-17f17d4c9250\") "
Jul 24 15:50:57.720297 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.720273    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:57.720414 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.720369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-console-config" (OuterVolumeSpecName: "console-config") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:57.720661 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.720417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:57.720661 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.720629    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-service-ca" (OuterVolumeSpecName: "service-ca") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:57.722300 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.722269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:50:57.722300 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.722283    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57fe378f-7b4e-46be-a231-17f17d4c9250-kube-api-access-7wm5p" (OuterVolumeSpecName: "kube-api-access-7wm5p") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "kube-api-access-7wm5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:50:57.722300 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.722296    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "57fe378f-7b4e-46be-a231-17f17d4c9250" (UID: "57fe378f-7b4e-46be-a231-17f17d4c9250"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:50:57.820964 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.820921    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-trusted-ca-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:57.820964 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.820954    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wm5p\" (UniqueName: \"kubernetes.io/projected/57fe378f-7b4e-46be-a231-17f17d4c9250-kube-api-access-7wm5p\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:57.820964 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.820964    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-console-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:57.821214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.820975    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:57.821214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.820995    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:57.821214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.821003    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57fe378f-7b4e-46be-a231-17f17d4c9250-oauth-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:57.821214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:57.821011    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57fe378f-7b4e-46be-a231-17f17d4c9250-console-oauth-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:50:58.429011 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.428936    2577 generic.go:356] "Generic (PLEG): container finished" podID="57fe378f-7b4e-46be-a231-17f17d4c9250" containerID="64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c" exitCode=0
Jul 24 15:50:58.429011 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.429011    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5864667bff-xqw8g"
Jul 24 15:50:58.429553 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.429022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5864667bff-xqw8g" event={"ID":"57fe378f-7b4e-46be-a231-17f17d4c9250","Type":"ContainerDied","Data":"64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c"}
Jul 24 15:50:58.429553 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.429077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5864667bff-xqw8g" event={"ID":"57fe378f-7b4e-46be-a231-17f17d4c9250","Type":"ContainerDied","Data":"28722ee3ebe932b364d4f78078f0db2859a81a096566f5f9f991383bd659c5fc"}
Jul 24 15:50:58.429553 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.429097    2577 scope.go:117] "RemoveContainer" containerID="64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c"
Jul 24 15:50:58.436876 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.436854    2577 scope.go:117] "RemoveContainer" containerID="64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c"
Jul 24 15:50:58.437192 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:50:58.437172    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c\": container with ID starting with 64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c not found: ID does not exist" containerID="64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c"
Jul 24 15:50:58.437243 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.437203    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c"} err="failed to get container status \"64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c\": rpc error: code = NotFound desc = could not find container \"64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c\": container with ID starting with 64d08f11c2af42899f3c3308235917cd4765374c9fa67c438448a55bb2b1247c not found: ID does not exist"
Jul 24 15:50:58.443662 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.443635    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5864667bff-xqw8g"]
Jul 24 15:50:58.447309 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:50:58.447284    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5864667bff-xqw8g"]
Jul 24 15:51:00.236974 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:00.236933    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57fe378f-7b4e-46be-a231-17f17d4c9250" path="/var/lib/kubelet/pods/57fe378f-7b4e-46be-a231-17f17d4c9250/volumes"
Jul 24 15:51:07.288655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:07.288593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:08.710332 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:08.710292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:08.771779 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:08.771744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:35.398352 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:35.398316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:38.311206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.311164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:38.413112 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.413075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:38.911382 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.911347    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"]
Jul 24 15:51:38.911682 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.911670    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57fe378f-7b4e-46be-a231-17f17d4c9250" containerName="console"
Jul 24 15:51:38.911743 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.911683    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="57fe378f-7b4e-46be-a231-17f17d4c9250" containerName="console"
Jul 24 15:51:38.911743 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.911731    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="57fe378f-7b4e-46be-a231-17f17d4c9250" containerName="console"
Jul 24 15:51:38.913955 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.913938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:38.916014 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.915995    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 15:51:38.916414 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.916397    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hnbtk\""
Jul 24 15:51:38.916414 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.916406    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 15:51:38.922704 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.922683    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"]
Jul 24 15:51:38.948832 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.948790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:38.948832 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.948835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lmx\" (UniqueName: \"kubernetes.io/projected/931c35c2-6e82-4421-88ea-8977bae7b8a4-kube-api-access-h2lmx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:38.949070 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:38.948932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.050164 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.050123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.050164 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.050165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lmx\" (UniqueName: \"kubernetes.io/projected/931c35c2-6e82-4421-88ea-8977bae7b8a4-kube-api-access-h2lmx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.050466 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.050206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.050654 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.050597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.050720 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.050673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.058069 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.058051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lmx\" (UniqueName: \"kubernetes.io/projected/931c35c2-6e82-4421-88ea-8977bae7b8a4-kube-api-access-h2lmx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.224420 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.224311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:39.350688 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.350659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"]
Jul 24 15:51:39.353015 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:51:39.352976    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931c35c2_6e82_4421_88ea_8977bae7b8a4.slice/crio-9f85cb800702a97860049b3043c2fa2007a0bb03f0c09bc23e19c4b8379662b3 WatchSource:0}: Error finding container 9f85cb800702a97860049b3043c2fa2007a0bb03f0c09bc23e19c4b8379662b3: Status 404 returned error can't find the container with id 9f85cb800702a97860049b3043c2fa2007a0bb03f0c09bc23e19c4b8379662b3
Jul 24 15:51:39.587957 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:39.587924    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf" event={"ID":"931c35c2-6e82-4421-88ea-8977bae7b8a4","Type":"ContainerStarted","Data":"9f85cb800702a97860049b3043c2fa2007a0bb03f0c09bc23e19c4b8379662b3"}
Jul 24 15:51:45.617889 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:45.617839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf" event={"ID":"931c35c2-6e82-4421-88ea-8977bae7b8a4","Type":"ContainerStarted","Data":"e3295231e9ce541630dd3432dffb3b634297b100e8b1203f38046b33c4623426"}
Jul 24 15:51:46.623133 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:46.623097    2577 generic.go:356] "Generic (PLEG): container finished" podID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerID="e3295231e9ce541630dd3432dffb3b634297b100e8b1203f38046b33c4623426" exitCode=0
Jul 24 15:51:46.623544 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:46.623142    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf" event={"ID":"931c35c2-6e82-4421-88ea-8977bae7b8a4","Type":"ContainerDied","Data":"e3295231e9ce541630dd3432dffb3b634297b100e8b1203f38046b33c4623426"}
Jul 24 15:51:47.542640 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:47.542582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:49.636711 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:49.636678    2577 generic.go:356] "Generic (PLEG): container finished" podID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerID="fbb8642617581cc5bebb30e376a457ef9032368d36afd5a772ef9d18aad70d12" exitCode=0
Jul 24 15:51:49.637108 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:49.636743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf" event={"ID":"931c35c2-6e82-4421-88ea-8977bae7b8a4","Type":"ContainerDied","Data":"fbb8642617581cc5bebb30e376a457ef9032368d36afd5a772ef9d18aad70d12"}
Jul 24 15:51:56.667265 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:56.667175    2577 generic.go:356] "Generic (PLEG): container finished" podID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerID="098a326b4bb8949ab903c900ecc0e0756d8046a2e2432578212a610c5100272d" exitCode=0
Jul 24 15:51:56.667265 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:56.667211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf" event={"ID":"931c35c2-6e82-4421-88ea-8977bae7b8a4","Type":"ContainerDied","Data":"098a326b4bb8949ab903c900ecc0e0756d8046a2e2432578212a610c5100272d"}
Jul 24 15:51:57.795715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.795690    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:57.920186 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.920137    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-util\") pod \"931c35c2-6e82-4421-88ea-8977bae7b8a4\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") "
Jul 24 15:51:57.920186 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.920191    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2lmx\" (UniqueName: \"kubernetes.io/projected/931c35c2-6e82-4421-88ea-8977bae7b8a4-kube-api-access-h2lmx\") pod \"931c35c2-6e82-4421-88ea-8977bae7b8a4\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") "
Jul 24 15:51:57.920454 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.920214    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-bundle\") pod \"931c35c2-6e82-4421-88ea-8977bae7b8a4\" (UID: \"931c35c2-6e82-4421-88ea-8977bae7b8a4\") "
Jul 24 15:51:57.920856 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.920829    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-bundle" (OuterVolumeSpecName: "bundle") pod "931c35c2-6e82-4421-88ea-8977bae7b8a4" (UID: "931c35c2-6e82-4421-88ea-8977bae7b8a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:51:57.922410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.922360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931c35c2-6e82-4421-88ea-8977bae7b8a4-kube-api-access-h2lmx" (OuterVolumeSpecName: "kube-api-access-h2lmx") pod "931c35c2-6e82-4421-88ea-8977bae7b8a4" (UID: "931c35c2-6e82-4421-88ea-8977bae7b8a4"). InnerVolumeSpecName "kube-api-access-h2lmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:51:57.924350 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:57.924328    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-util" (OuterVolumeSpecName: "util") pod "931c35c2-6e82-4421-88ea-8977bae7b8a4" (UID: "931c35c2-6e82-4421-88ea-8977bae7b8a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:51:58.021294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.021257    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:51:58.021294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.021290    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lmx\" (UniqueName: \"kubernetes.io/projected/931c35c2-6e82-4421-88ea-8977bae7b8a4-kube-api-access-h2lmx\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:51:58.021294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.021300    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/931c35c2-6e82-4421-88ea-8977bae7b8a4-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:51:58.125234 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.125188    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 15:51:58.677277 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.677233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf" event={"ID":"931c35c2-6e82-4421-88ea-8977bae7b8a4","Type":"ContainerDied","Data":"9f85cb800702a97860049b3043c2fa2007a0bb03f0c09bc23e19c4b8379662b3"}
Jul 24 15:51:58.677277 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.677286    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f85cb800702a97860049b3043c2fa2007a0bb03f0c09bc23e19c4b8379662b3"
Jul 24 15:51:58.677513 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:58.677255    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rznnf"
Jul 24 15:51:59.902446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:51:59.902407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:00.687761 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:00.687725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:01.859394 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859359    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"]
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859684    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="extract"
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859696    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="extract"
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859718    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="pull"
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859723    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="pull"
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859732    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="util"
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859737    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="util"
Jul 24 15:52:01.859786 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.859782    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="931c35c2-6e82-4421-88ea-8977bae7b8a4" containerName="extract"
Jul 24 15:52:01.913587 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.913541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"]
Jul 24 15:52:01.913831 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.913696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:01.916184 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.916162    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-r8t6j\""
Jul 24 15:52:01.916595 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.916579    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:52:01.916760 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:01.916746    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 24 15:52:02.054315 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.054281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983c616c-b81a-4a3e-a394-0b4ed1650570-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hxxt\" (UID: \"983c616c-b81a-4a3e-a394-0b4ed1650570\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.054496 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.054338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvpg8\" (UniqueName: \"kubernetes.io/projected/983c616c-b81a-4a3e-a394-0b4ed1650570-kube-api-access-bvpg8\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hxxt\" (UID: \"983c616c-b81a-4a3e-a394-0b4ed1650570\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.155769 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.155698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvpg8\" (UniqueName: \"kubernetes.io/projected/983c616c-b81a-4a3e-a394-0b4ed1650570-kube-api-access-bvpg8\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hxxt\" (UID: \"983c616c-b81a-4a3e-a394-0b4ed1650570\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.155889 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.155770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983c616c-b81a-4a3e-a394-0b4ed1650570-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hxxt\" (UID: \"983c616c-b81a-4a3e-a394-0b4ed1650570\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.156081 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.156066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983c616c-b81a-4a3e-a394-0b4ed1650570-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hxxt\" (UID: \"983c616c-b81a-4a3e-a394-0b4ed1650570\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.163659 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.163639    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvpg8\" (UniqueName: \"kubernetes.io/projected/983c616c-b81a-4a3e-a394-0b4ed1650570-kube-api-access-bvpg8\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hxxt\" (UID: \"983c616c-b81a-4a3e-a394-0b4ed1650570\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.222592 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.222568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"
Jul 24 15:52:02.373969 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.373947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt"]
Jul 24 15:52:02.376790 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:02.376758    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983c616c_b81a_4a3e_a394_0b4ed1650570.slice/crio-bfda4ee9246fa601f370542a69ce1c1942138564a6c5b124f525b6d36c98b80d WatchSource:0}: Error finding container bfda4ee9246fa601f370542a69ce1c1942138564a6c5b124f525b6d36c98b80d: Status 404 returned error can't find the container with id bfda4ee9246fa601f370542a69ce1c1942138564a6c5b124f525b6d36c98b80d
Jul 24 15:52:02.379548 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.379530    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:52:02.694736 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:02.694681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt" event={"ID":"983c616c-b81a-4a3e-a394-0b4ed1650570","Type":"ContainerStarted","Data":"bfda4ee9246fa601f370542a69ce1c1942138564a6c5b124f525b6d36c98b80d"}
Jul 24 15:52:03.514025 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:03.513986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:03.615130 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:03.615097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:05.284877 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.284844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:05.384377 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.384343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:05.473658 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.473608    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wxp85"]
Jul 24 15:52:05.494345 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.494316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.496387 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.496361    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wxp85"]
Jul 24 15:52:05.499258 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.499239    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 15:52:05.499480 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.499465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-s8p6h\""
Jul 24 15:52:05.500664 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.500645    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 15:52:05.582978 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.582891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3d670d2-10da-479c-a74e-d474c11a5be5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wxp85\" (UID: \"b3d670d2-10da-479c-a74e-d474c11a5be5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.582978 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.582937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cwj7\" (UniqueName: \"kubernetes.io/projected/b3d670d2-10da-479c-a74e-d474c11a5be5-kube-api-access-7cwj7\") pod \"cert-manager-webhook-6486ffd997-wxp85\" (UID: \"b3d670d2-10da-479c-a74e-d474c11a5be5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.683689 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.683648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3d670d2-10da-479c-a74e-d474c11a5be5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wxp85\" (UID: \"b3d670d2-10da-479c-a74e-d474c11a5be5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.683689 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.683696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cwj7\" (UniqueName: \"kubernetes.io/projected/b3d670d2-10da-479c-a74e-d474c11a5be5-kube-api-access-7cwj7\") pod \"cert-manager-webhook-6486ffd997-wxp85\" (UID: \"b3d670d2-10da-479c-a74e-d474c11a5be5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.691008 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.690971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3d670d2-10da-479c-a74e-d474c11a5be5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wxp85\" (UID: \"b3d670d2-10da-479c-a74e-d474c11a5be5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.691135 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.691097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cwj7\" (UniqueName: \"kubernetes.io/projected/b3d670d2-10da-479c-a74e-d474c11a5be5-kube-api-access-7cwj7\") pod \"cert-manager-webhook-6486ffd997-wxp85\" (UID: \"b3d670d2-10da-479c-a74e-d474c11a5be5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.711759 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.711727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt" event={"ID":"983c616c-b81a-4a3e-a394-0b4ed1650570","Type":"ContainerStarted","Data":"a95e6501c51c3861fa3d1151fabf2a2d463798cbc8be8a102454f284abe41379"}
Jul 24 15:52:05.729585 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.729537    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hxxt" podStartSLOduration=2.5010882 podStartE2EDuration="4.729522934s" totalImagesPullingTime="2.228434734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:05.727772791 +0000 UTC m=+308.055337329" watchObservedRunningTime="2026-07-24 15:52:05.729522934 +0000 UTC m=+308.057087473"
Jul 24 15:52:05.822587 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.822553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:05.967149 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:05.967116    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wxp85"]
Jul 24 15:52:05.970225 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:05.970197    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3d670d2_10da_479c_a74e_d474c11a5be5.slice/crio-b737834eba8e5791e5a8959f135327ae4883484a8e57432f9ef760944b730381 WatchSource:0}: Error finding container b737834eba8e5791e5a8959f135327ae4883484a8e57432f9ef760944b730381: Status 404 returned error can't find the container with id b737834eba8e5791e5a8959f135327ae4883484a8e57432f9ef760944b730381
Jul 24 15:52:06.147105 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.146970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:06.257433 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.257398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:06.336850 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.336816    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"]
Jul 24 15:52:06.339709 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.339685    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.341781 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.341755    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 15:52:06.342118 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.342075    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 15:52:06.342219 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.342154    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hnbtk\""
Jul 24 15:52:06.354485 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.354436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"]
Jul 24 15:52:06.496048 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.495937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdlq6\" (UniqueName: \"kubernetes.io/projected/77aaf582-4c1f-44da-abea-8ec39870eee3-kube-api-access-jdlq6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.496220 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.496066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.496220 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.496120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.596913 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.596868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.597133 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.596937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.597133 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.596980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdlq6\" (UniqueName: \"kubernetes.io/projected/77aaf582-4c1f-44da-abea-8ec39870eee3-kube-api-access-jdlq6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.597348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.597327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.597415 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.597343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.605245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.605211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdlq6\" (UniqueName: \"kubernetes.io/projected/77aaf582-4c1f-44da-abea-8ec39870eee3-kube-api-access-jdlq6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.649939 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.649901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:06.718277 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.718243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85" event={"ID":"b3d670d2-10da-479c-a74e-d474c11a5be5","Type":"ContainerStarted","Data":"b737834eba8e5791e5a8959f135327ae4883484a8e57432f9ef760944b730381"}
Jul 24 15:52:06.790986 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:06.790945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"]
Jul 24 15:52:06.794032 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:06.794003    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77aaf582_4c1f_44da_abea_8ec39870eee3.slice/crio-9c38f7033913228c9b7f6d4c9bb0b99c7c25351af74e00e9ae2c4fa97b4b0f7a WatchSource:0}: Error finding container 9c38f7033913228c9b7f6d4c9bb0b99c7c25351af74e00e9ae2c4fa97b4b0f7a: Status 404 returned error can't find the container with id 9c38f7033913228c9b7f6d4c9bb0b99c7c25351af74e00e9ae2c4fa97b4b0f7a
Jul 24 15:52:07.322322 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:07.322274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:07.682940 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:07.682855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:07.727978 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:07.727923    2577 generic.go:356] "Generic (PLEG): container finished" podID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerID="0ac01ef7aaebb6e85a3ac69497c50cc9a8ede22cc05698166729c11783080964" exitCode=0
Jul 24 15:52:07.727978 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:07.727977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp" event={"ID":"77aaf582-4c1f-44da-abea-8ec39870eee3","Type":"ContainerDied","Data":"0ac01ef7aaebb6e85a3ac69497c50cc9a8ede22cc05698166729c11783080964"}
Jul 24 15:52:07.728228 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:07.728024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp" event={"ID":"77aaf582-4c1f-44da-abea-8ec39870eee3","Type":"ContainerStarted","Data":"9c38f7033913228c9b7f6d4c9bb0b99c7c25351af74e00e9ae2c4fa97b4b0f7a"}
Jul 24 15:52:09.743059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:09.743003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85" event={"ID":"b3d670d2-10da-479c-a74e-d474c11a5be5","Type":"ContainerStarted","Data":"fb537c5d5aeb494d3c8d45c65c8201292d8aab675b98c40e4d3352096b9c4cc3"}
Jul 24 15:52:09.743674 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:09.743581    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:09.760891 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:09.760812    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85" podStartSLOduration=1.916501853 podStartE2EDuration="4.760789009s" totalImagesPullingTime="2.844287156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:09.758863677 +0000 UTC m=+312.086428285" watchObservedRunningTime="2026-07-24 15:52:09.760789009 +0000 UTC m=+312.088353548"
Jul 24 15:52:10.540912 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:10.540871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:10.640025 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:10.639980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:10.750086 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:10.750000    2577 generic.go:356] "Generic (PLEG): container finished" podID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerID="bd7c7b95412a49f5dca73d655f7b5cde20c538853fbb3a17ad041746bc95fcbc" exitCode=0
Jul 24 15:52:10.750086 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:10.750076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp" event={"ID":"77aaf582-4c1f-44da-abea-8ec39870eee3","Type":"ContainerDied","Data":"bd7c7b95412a49f5dca73d655f7b5cde20c538853fbb3a17ad041746bc95fcbc"}
Jul 24 15:52:11.756646 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:11.756594    2577 generic.go:356] "Generic (PLEG): container finished" podID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerID="43baf0dad5b488e5d5f39cb80e266aab06cd25892fca2745fdf8523ddb1d3e16" exitCode=0
Jul 24 15:52:11.757162 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:11.756652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp" event={"ID":"77aaf582-4c1f-44da-abea-8ec39870eee3","Type":"ContainerDied","Data":"43baf0dad5b488e5d5f39cb80e266aab06cd25892fca2745fdf8523ddb1d3e16"}
Jul 24 15:52:12.896897 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:12.896870    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:13.055564 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.055526    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-bundle\") pod \"77aaf582-4c1f-44da-abea-8ec39870eee3\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") "
Jul 24 15:52:13.055762 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.055575    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-util\") pod \"77aaf582-4c1f-44da-abea-8ec39870eee3\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") "
Jul 24 15:52:13.055762 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.055660    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdlq6\" (UniqueName: \"kubernetes.io/projected/77aaf582-4c1f-44da-abea-8ec39870eee3-kube-api-access-jdlq6\") pod \"77aaf582-4c1f-44da-abea-8ec39870eee3\" (UID: \"77aaf582-4c1f-44da-abea-8ec39870eee3\") "
Jul 24 15:52:13.055972 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.055945    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-bundle" (OuterVolumeSpecName: "bundle") pod "77aaf582-4c1f-44da-abea-8ec39870eee3" (UID: "77aaf582-4c1f-44da-abea-8ec39870eee3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:52:13.057765 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.057744    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77aaf582-4c1f-44da-abea-8ec39870eee3-kube-api-access-jdlq6" (OuterVolumeSpecName: "kube-api-access-jdlq6") pod "77aaf582-4c1f-44da-abea-8ec39870eee3" (UID: "77aaf582-4c1f-44da-abea-8ec39870eee3"). InnerVolumeSpecName "kube-api-access-jdlq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:52:13.060422 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.060398    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-util" (OuterVolumeSpecName: "util") pod "77aaf582-4c1f-44da-abea-8ec39870eee3" (UID: "77aaf582-4c1f-44da-abea-8ec39870eee3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:52:13.067906 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.067879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:13.129519 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.129485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:13.156206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.156168    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:52:13.156206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.156200    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aaf582-4c1f-44da-abea-8ec39870eee3-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:52:13.156206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.156210    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdlq6\" (UniqueName: \"kubernetes.io/projected/77aaf582-4c1f-44da-abea-8ec39870eee3-kube-api-access-jdlq6\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:52:13.766792 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.766756    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp"
Jul 24 15:52:13.766985 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.766766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f47nqp" event={"ID":"77aaf582-4c1f-44da-abea-8ec39870eee3","Type":"ContainerDied","Data":"9c38f7033913228c9b7f6d4c9bb0b99c7c25351af74e00e9ae2c4fa97b4b0f7a"}
Jul 24 15:52:13.766985 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:13.766877    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c38f7033913228c9b7f6d4c9bb0b99c7c25351af74e00e9ae2c4fa97b4b0f7a"
Jul 24 15:52:15.885077 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:15.885043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:16.759181 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:16.759142    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wxp85"
Jul 24 15:52:18.119957 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.119916    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cmwlc"]
Jul 24 15:52:18.120439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120329    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="extract"
Jul 24 15:52:18.120439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120345    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="extract"
Jul 24 15:52:18.120439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120364    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="pull"
Jul 24 15:52:18.120439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120372    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="pull"
Jul 24 15:52:18.120439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120404    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="util"
Jul 24 15:52:18.120439 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120415    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="util"
Jul 24 15:52:18.120770 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.120488    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="77aaf582-4c1f-44da-abea-8ec39870eee3" containerName="extract"
Jul 24 15:52:18.124136 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.124115    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.126121 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.126100    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-tz564\""
Jul 24 15:52:18.131935 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.131906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cmwlc"]
Jul 24 15:52:18.174990 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.174953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:18.280815 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.280781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:18.299878 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.299821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrbc4\" (UniqueName: \"kubernetes.io/projected/d845b86a-cc1b-47c7-95b7-3739a4f01db1-kube-api-access-qrbc4\") pod \"cert-manager-55b4d954d5-cmwlc\" (UID: \"d845b86a-cc1b-47c7-95b7-3739a4f01db1\") " pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.300035 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.299894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d845b86a-cc1b-47c7-95b7-3739a4f01db1-bound-sa-token\") pod \"cert-manager-55b4d954d5-cmwlc\" (UID: \"d845b86a-cc1b-47c7-95b7-3739a4f01db1\") " pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.400540 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.400444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrbc4\" (UniqueName: \"kubernetes.io/projected/d845b86a-cc1b-47c7-95b7-3739a4f01db1-kube-api-access-qrbc4\") pod \"cert-manager-55b4d954d5-cmwlc\" (UID: \"d845b86a-cc1b-47c7-95b7-3739a4f01db1\") " pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.400540 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.400494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d845b86a-cc1b-47c7-95b7-3739a4f01db1-bound-sa-token\") pod \"cert-manager-55b4d954d5-cmwlc\" (UID: \"d845b86a-cc1b-47c7-95b7-3739a4f01db1\") " pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.407904 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.407874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d845b86a-cc1b-47c7-95b7-3739a4f01db1-bound-sa-token\") pod \"cert-manager-55b4d954d5-cmwlc\" (UID: \"d845b86a-cc1b-47c7-95b7-3739a4f01db1\") " pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.408054 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.408030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrbc4\" (UniqueName: \"kubernetes.io/projected/d845b86a-cc1b-47c7-95b7-3739a4f01db1-kube-api-access-qrbc4\") pod \"cert-manager-55b4d954d5-cmwlc\" (UID: \"d845b86a-cc1b-47c7-95b7-3739a4f01db1\") " pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.433753 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.433706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cmwlc"
Jul 24 15:52:18.568682 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.568647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cmwlc"]
Jul 24 15:52:18.572561 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:18.572532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd845b86a_cc1b_47c7_95b7_3739a4f01db1.slice/crio-25c2a19d11a7c7c8ba478d1a610c86138a249d1c82419c78a971ea03150fff9c WatchSource:0}: Error finding container 25c2a19d11a7c7c8ba478d1a610c86138a249d1c82419c78a971ea03150fff9c: Status 404 returned error can't find the container with id 25c2a19d11a7c7c8ba478d1a610c86138a249d1c82419c78a971ea03150fff9c
Jul 24 15:52:18.789266 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.789230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cmwlc" event={"ID":"d845b86a-cc1b-47c7-95b7-3739a4f01db1","Type":"ContainerStarted","Data":"e8e47f6260f60a07d6757fc358ff09f37fa43f848d45c0e7d6f2b9868c46b524"}
Jul 24 15:52:18.789443 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.789274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cmwlc" event={"ID":"d845b86a-cc1b-47c7-95b7-3739a4f01db1","Type":"ContainerStarted","Data":"25c2a19d11a7c7c8ba478d1a610c86138a249d1c82419c78a971ea03150fff9c"}
Jul 24 15:52:18.805639 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:18.805568    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cmwlc" podStartSLOduration=0.805551449 podStartE2EDuration="805.551449ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:18.804409499 +0000 UTC m=+321.131974037" watchObservedRunningTime="2026-07-24 15:52:18.805551449 +0000 UTC m=+321.133115986"
Jul 24 15:52:20.019286 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.019241    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"]
Jul 24 15:52:20.024425 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.024399    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.026300 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.026267    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 15:52:20.026648 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.026628    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-l9rkl\""
Jul 24 15:52:20.026748 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.026632    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:52:20.033400 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.033374    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"]
Jul 24 15:52:20.113055 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.113017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkbc9\" (UniqueName: \"kubernetes.io/projected/47357e38-5ac3-4ea2-b7ba-039a343bde53-kube-api-access-kkbc9\") pod \"openshift-lws-operator-fd8ccff4c-j9v6g\" (UID: \"47357e38-5ac3-4ea2-b7ba-039a343bde53\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.113250 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.113074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47357e38-5ac3-4ea2-b7ba-039a343bde53-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j9v6g\" (UID: \"47357e38-5ac3-4ea2-b7ba-039a343bde53\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.213573 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.213538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkbc9\" (UniqueName: \"kubernetes.io/projected/47357e38-5ac3-4ea2-b7ba-039a343bde53-kube-api-access-kkbc9\") pod \"openshift-lws-operator-fd8ccff4c-j9v6g\" (UID: \"47357e38-5ac3-4ea2-b7ba-039a343bde53\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.213770 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.213597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47357e38-5ac3-4ea2-b7ba-039a343bde53-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j9v6g\" (UID: \"47357e38-5ac3-4ea2-b7ba-039a343bde53\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.214047 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.214029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47357e38-5ac3-4ea2-b7ba-039a343bde53-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j9v6g\" (UID: \"47357e38-5ac3-4ea2-b7ba-039a343bde53\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.221206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.221180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkbc9\" (UniqueName: \"kubernetes.io/projected/47357e38-5ac3-4ea2-b7ba-039a343bde53-kube-api-access-kkbc9\") pod \"openshift-lws-operator-fd8ccff4c-j9v6g\" (UID: \"47357e38-5ac3-4ea2-b7ba-039a343bde53\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.334965 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.334924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"
Jul 24 15:52:20.463134 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.463100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g"]
Jul 24 15:52:20.466760 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:20.466732    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47357e38_5ac3_4ea2_b7ba_039a343bde53.slice/crio-096ac4cf871d878507a6024e75b4050bdb35462d57ed488c9288908ee7ad6546 WatchSource:0}: Error finding container 096ac4cf871d878507a6024e75b4050bdb35462d57ed488c9288908ee7ad6546: Status 404 returned error can't find the container with id 096ac4cf871d878507a6024e75b4050bdb35462d57ed488c9288908ee7ad6546
Jul 24 15:52:20.799854 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:20.799818    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g" event={"ID":"47357e38-5ac3-4ea2-b7ba-039a343bde53","Type":"ContainerStarted","Data":"096ac4cf871d878507a6024e75b4050bdb35462d57ed488c9288908ee7ad6546"}
Jul 24 15:52:39.884488 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:39.884450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g" event={"ID":"47357e38-5ac3-4ea2-b7ba-039a343bde53","Type":"ContainerStarted","Data":"7e6d2ec7eac0878734b42eea17cf25cf8375be110693c26a8f452dcfe41d2a4a"}
Jul 24 15:52:39.900138 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:39.900084    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9v6g" podStartSLOduration=0.678271808 podStartE2EDuration="19.900067773s" totalImagesPullingTime="19.221795965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:39.898384109 +0000 UTC m=+342.225948638" watchObservedRunningTime="2026-07-24 15:52:39.900067773 +0000 UTC m=+342.227632310"
Jul 24 15:52:40.722401 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:40.722360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:41.550605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.550572    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"]
Jul 24 15:52:41.553097 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.553075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.554912 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.554893    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 15:52:41.555013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.554931    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hnbtk\""
Jul 24 15:52:41.555412 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.555391    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 15:52:41.561299 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.561277    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"]
Jul 24 15:52:41.579810 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.579785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.579898 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.579849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5h59\" (UniqueName: \"kubernetes.io/projected/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-kube-api-access-d5h59\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.579898 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.579887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.680947 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.680920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.681065 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.680962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.681065 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.681018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5h59\" (UniqueName: \"kubernetes.io/projected/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-kube-api-access-d5h59\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.681327 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.681309    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.681387 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.681356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.688034 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.688005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5h59\" (UniqueName: \"kubernetes.io/projected/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-kube-api-access-d5h59\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.857770 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.857712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:41.862495 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.862478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:41.983721 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:41.983693    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"]
Jul 24 15:52:41.985728 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:41.985702    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91ae5c7_c2f7_42d9_b1da_da52e6a096ec.slice/crio-1faaee74ce26d865a3e762b550466206408058a67359ed75787043f4bb1a109e WatchSource:0}: Error finding container 1faaee74ce26d865a3e762b550466206408058a67359ed75787043f4bb1a109e: Status 404 returned error can't find the container with id 1faaee74ce26d865a3e762b550466206408058a67359ed75787043f4bb1a109e
Jul 24 15:52:42.900049 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:42.900015    2577 generic.go:356] "Generic (PLEG): container finished" podID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerID="e602b191bfa1914dd720634878fd4303eaa3d07814c8432c39f1704b66d85556" exitCode=0
Jul 24 15:52:42.900420 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:42.900099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz" event={"ID":"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec","Type":"ContainerDied","Data":"e602b191bfa1914dd720634878fd4303eaa3d07814c8432c39f1704b66d85556"}
Jul 24 15:52:42.900420 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:42.900146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz" event={"ID":"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec","Type":"ContainerStarted","Data":"1faaee74ce26d865a3e762b550466206408058a67359ed75787043f4bb1a109e"}
Jul 24 15:52:43.014895 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:43.014815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:43.906438 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:43.906404    2577 generic.go:356] "Generic (PLEG): container finished" podID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerID="95371347eecf0b9db24b303a583d5c2b65a9f34c445f3403272441bf33d1ccf6" exitCode=0
Jul 24 15:52:43.906869 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:43.906492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz" event={"ID":"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec","Type":"ContainerDied","Data":"95371347eecf0b9db24b303a583d5c2b65a9f34c445f3403272441bf33d1ccf6"}
Jul 24 15:52:44.912317 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:44.912282    2577 generic.go:356] "Generic (PLEG): container finished" podID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerID="edccba5e063254f2f6958d1d9508d7c23fa267f30b6e1d4cfad4c8aa080cd5ac" exitCode=0
Jul 24 15:52:44.912750 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:44.912325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz" event={"ID":"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec","Type":"ContainerDied","Data":"edccba5e063254f2f6958d1d9508d7c23fa267f30b6e1d4cfad4c8aa080cd5ac"}
Jul 24 15:52:46.049005 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.048976    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:46.119200 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.119174    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-bundle\") pod \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") "
Jul 24 15:52:46.119332 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.119210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-util\") pod \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") "
Jul 24 15:52:46.119332 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.119256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5h59\" (UniqueName: \"kubernetes.io/projected/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-kube-api-access-d5h59\") pod \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\" (UID: \"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec\") "
Jul 24 15:52:46.119848 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.119828    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-bundle" (OuterVolumeSpecName: "bundle") pod "a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" (UID: "a91ae5c7-c2f7-42d9-b1da-da52e6a096ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:52:46.121309 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.121289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-kube-api-access-d5h59" (OuterVolumeSpecName: "kube-api-access-d5h59") pod "a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" (UID: "a91ae5c7-c2f7-42d9-b1da-da52e6a096ec"). InnerVolumeSpecName "kube-api-access-d5h59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:52:46.124305 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.124273    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-util" (OuterVolumeSpecName: "util") pod "a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" (UID: "a91ae5c7-c2f7-42d9-b1da-da52e6a096ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:52:46.220924 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.220851    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:52:46.220924 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.220877    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:52:46.220924 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.220887    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5h59\" (UniqueName: \"kubernetes.io/projected/a91ae5c7-c2f7-42d9-b1da-da52e6a096ec-kube-api-access-d5h59\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:52:46.923731 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.923702    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz"
Jul 24 15:52:46.923871 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.923690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7jlsz" event={"ID":"a91ae5c7-c2f7-42d9-b1da-da52e6a096ec","Type":"ContainerDied","Data":"1faaee74ce26d865a3e762b550466206408058a67359ed75787043f4bb1a109e"}
Jul 24 15:52:46.923871 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:46.923819    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1faaee74ce26d865a3e762b550466206408058a67359ed75787043f4bb1a109e"
Jul 24 15:52:47.227647 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.227533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:47.294973 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.294938    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"]
Jul 24 15:52:47.295239 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295227    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="pull"
Jul 24 15:52:47.295291 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295240    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="pull"
Jul 24 15:52:47.295291 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295252    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="extract"
Jul 24 15:52:47.295291 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295258    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="extract"
Jul 24 15:52:47.295291 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295267    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="util"
Jul 24 15:52:47.295291 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295272    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="util"
Jul 24 15:52:47.295453 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.295332    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91ae5c7-c2f7-42d9-b1da-da52e6a096ec" containerName="extract"
Jul 24 15:52:47.298605 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.298587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.304218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.304188    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ftnp6\""
Jul 24 15:52:47.305015 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.304987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 15:52:47.306106 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.306090    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 15:52:47.306486 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.306469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 15:52:47.321075 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.321046    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"]
Jul 24 15:52:47.327709 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.327672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:47.331472 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.331452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fzz5\" (UniqueName: \"kubernetes.io/projected/b0c5880a-e163-4101-a031-8f2f302e1dc8-kube-api-access-7fzz5\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.331570 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.331519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0c5880a-e163-4101-a031-8f2f302e1dc8-manager-config\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.331656 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.331590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0c5880a-e163-4101-a031-8f2f302e1dc8-metrics-cert\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.331707 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.331673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0c5880a-e163-4101-a031-8f2f302e1dc8-cert\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.432892 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.432856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0c5880a-e163-4101-a031-8f2f302e1dc8-metrics-cert\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.433095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.432911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0c5880a-e163-4101-a031-8f2f302e1dc8-cert\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.433095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.432941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fzz5\" (UniqueName: \"kubernetes.io/projected/b0c5880a-e163-4101-a031-8f2f302e1dc8-kube-api-access-7fzz5\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.433095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.432962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0c5880a-e163-4101-a031-8f2f302e1dc8-manager-config\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.433715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.433690    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0c5880a-e163-4101-a031-8f2f302e1dc8-manager-config\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.435348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.435329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0c5880a-e163-4101-a031-8f2f302e1dc8-metrics-cert\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.435518 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.435497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0c5880a-e163-4101-a031-8f2f302e1dc8-cert\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.440729 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.440701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fzz5\" (UniqueName: \"kubernetes.io/projected/b0c5880a-e163-4101-a031-8f2f302e1dc8-kube-api-access-7fzz5\") pod \"lws-controller-manager-777c588886-w7ntt\" (UID: \"b0c5880a-e163-4101-a031-8f2f302e1dc8\") " pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.607310 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.607279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:47.732640 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.732602    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"]
Jul 24 15:52:47.735029 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:47.735000    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c5880a_e163_4101_a031_8f2f302e1dc8.slice/crio-2d44f28e5c0f51ed856bf419e99468dbdb6ab2dfe0b8195acd7bccbe347ea707 WatchSource:0}: Error finding container 2d44f28e5c0f51ed856bf419e99468dbdb6ab2dfe0b8195acd7bccbe347ea707: Status 404 returned error can't find the container with id 2d44f28e5c0f51ed856bf419e99468dbdb6ab2dfe0b8195acd7bccbe347ea707
Jul 24 15:52:47.929534 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:47.929452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt" event={"ID":"b0c5880a-e163-4101-a031-8f2f302e1dc8","Type":"ContainerStarted","Data":"2d44f28e5c0f51ed856bf419e99468dbdb6ab2dfe0b8195acd7bccbe347ea707"}
Jul 24 15:52:49.939715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:49.939670    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt" event={"ID":"b0c5880a-e163-4101-a031-8f2f302e1dc8","Type":"ContainerStarted","Data":"f03cc37b5a051c357a32b9f52e5aafa42ef56f27005c06d185d4ec53cfe43e23"}
Jul 24 15:52:49.940162 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:49.939819    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:52:49.955248 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:49.955196    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt" podStartSLOduration=1.511588153 podStartE2EDuration="2.95517938s" totalImagesPullingTime="1.443591227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:49.954345101 +0000 UTC m=+352.281909640" watchObservedRunningTime="2026-07-24 15:52:49.95517938 +0000 UTC m=+352.282743919"
Jul 24 15:52:57.297298 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:57.297235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:57.408289 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:57.408248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:57.926165 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:57.926108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.128265 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.128205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.233397 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.233306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.303258 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.303219    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"]
Jul 24 15:52:58.306959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.306933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.313139 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.313109    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 15:52:58.313516 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.313109    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 15:52:58.317540 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.317515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 15:52:58.317941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.317920    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9glrs\""
Jul 24 15:52:58.318348 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.318331    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 15:52:58.330967 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.330929    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"]
Jul 24 15:52:58.424202 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.424164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9w9l\" (UniqueName: \"kubernetes.io/projected/08737d3f-3b02-4c1e-a82d-c70113b2f206-kube-api-access-d9w9l\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.424202 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.424206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08737d3f-3b02-4c1e-a82d-c70113b2f206-manifests\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.424410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.424243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08737d3f-3b02-4c1e-a82d-c70113b2f206-tmp\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.424410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.424355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08737d3f-3b02-4c1e-a82d-c70113b2f206-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.424410 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.424394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08737d3f-3b02-4c1e-a82d-c70113b2f206-webhook-cert\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.465787 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.465748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.525252 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.525215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08737d3f-3b02-4c1e-a82d-c70113b2f206-webhook-cert\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.525465 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.525291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9w9l\" (UniqueName: \"kubernetes.io/projected/08737d3f-3b02-4c1e-a82d-c70113b2f206-kube-api-access-d9w9l\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.525465 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.525322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08737d3f-3b02-4c1e-a82d-c70113b2f206-manifests\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.525465 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.525363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08737d3f-3b02-4c1e-a82d-c70113b2f206-tmp\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.525465 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.525425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08737d3f-3b02-4c1e-a82d-c70113b2f206-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.526124 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.526099    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08737d3f-3b02-4c1e-a82d-c70113b2f206-manifests\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.529013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.528954    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08737d3f-3b02-4c1e-a82d-c70113b2f206-tmp\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.529186 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.529040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08737d3f-3b02-4c1e-a82d-c70113b2f206-webhook-cert\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.529186 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.529040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08737d3f-3b02-4c1e-a82d-c70113b2f206-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.535218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.535176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9w9l\" (UniqueName: \"kubernetes.io/projected/08737d3f-3b02-4c1e-a82d-c70113b2f206-kube-api-access-d9w9l\") pod \"opendatahub-operator-controller-manager-749946f7d7-c4ps8\" (UID: \"08737d3f-3b02-4c1e-a82d-c70113b2f206\") " pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.568913 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.568873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.619033 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.618990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:52:58.666546 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.666490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.753660 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.753609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.768596 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.768574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"]
Jul 24 15:52:58.770919 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:52:58.770894    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08737d3f_3b02_4c1e_a82d_c70113b2f206.slice/crio-316c09748ed7a4e393b59ee3b646c39d27422490b2ae869349c76fde9011659b WatchSource:0}: Error finding container 316c09748ed7a4e393b59ee3b646c39d27422490b2ae869349c76fde9011659b: Status 404 returned error can't find the container with id 316c09748ed7a4e393b59ee3b646c39d27422490b2ae869349c76fde9011659b
Jul 24 15:52:58.817136 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.817072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.918030 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.917995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:58.988715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:58.988683    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8" event={"ID":"08737d3f-3b02-4c1e-a82d-c70113b2f206","Type":"ContainerStarted","Data":"316c09748ed7a4e393b59ee3b646c39d27422490b2ae869349c76fde9011659b"}
Jul 24 15:52:59.017276 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:52:59.017238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:00.949087 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:00.949053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-777c588886-w7ntt"
Jul 24 15:53:02.008236 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:02.008070    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8" event={"ID":"08737d3f-3b02-4c1e-a82d-c70113b2f206","Type":"ContainerStarted","Data":"e712c26d91629c6dc2712f821fe29595553d052be7ec1d0731c0b4e78302c91e"}
Jul 24 15:53:03.016177 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:03.016124    2577 generic.go:356] "Generic (PLEG): container finished" podID="08737d3f-3b02-4c1e-a82d-c70113b2f206" containerID="e712c26d91629c6dc2712f821fe29595553d052be7ec1d0731c0b4e78302c91e" exitCode=0
Jul 24 15:53:03.016643 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:03.016207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8" event={"ID":"08737d3f-3b02-4c1e-a82d-c70113b2f206","Type":"ContainerDied","Data":"e712c26d91629c6dc2712f821fe29595553d052be7ec1d0731c0b4e78302c91e"}
Jul 24 15:53:04.026737 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:04.026666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8" event={"ID":"08737d3f-3b02-4c1e-a82d-c70113b2f206","Type":"ContainerStarted","Data":"e2c0038060f8b1b913469d75c834c4f76590c0bd9b46422ab5ed2771eb0b3bc0"}
Jul 24 15:53:04.027446 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:04.026823    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:53:04.048262 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:04.048202    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8" podStartSLOduration=1.709935531 podStartE2EDuration="6.048181174s" totalImagesPullingTime="3.235593303s" totalInitContainerRuntime="1.10265234s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:53:04.046090991 +0000 UTC m=+366.373655548" watchObservedRunningTime="2026-07-24 15:53:04.048181174 +0000 UTC m=+366.375745712"
Jul 24 15:53:15.036143 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:15.036067    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749946f7d7-c4ps8"
Jul 24 15:53:17.133982 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:17.133909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:17.311680 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:17.311648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:17.954123 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:17.954084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:18.054454 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:18.054415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:18.155558 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:18.155504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:18.259917 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:18.259806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:18.555062 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:18.555030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:19.152156 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:19.152120    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:23.143258 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:23.143193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:23.304650 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:23.304602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:24.182225 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:24.182186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:24.255205 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:24.255166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:25.467600 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:25.467559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:26.775203 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.775139    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"]
Jul 24 15:53:26.779874 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.779840    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.782039 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.782013    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6rkhw\""
Jul 24 15:53:26.782200 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.782010    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 15:53:26.782980 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.782960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 15:53:26.782980 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.782977    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 15:53:26.783081 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.783022    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 15:53:26.791384 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.791348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"]
Jul 24 15:53:26.876920 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.876880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.877245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.876927    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.877245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.876957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.877245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.877015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz78c\" (UniqueName: \"kubernetes.io/projected/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-kube-api-access-qz78c\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.877245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.877070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.877245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.877127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.877245 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.877216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978181 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978391 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978391 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978391 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz78c\" (UniqueName: \"kubernetes.io/projected/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-kube-api-access-qz78c\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978550 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978550 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.978673 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.978548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.979120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.979089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.980690 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.980663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.980798 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.980692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.980865 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.980817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.980865 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.980824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.991414 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.991357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:26.991643 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:26.991534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz78c\" (UniqueName: \"kubernetes.io/projected/f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e-kube-api-access-qz78c\") pod \"istiod-openshift-gateway-5677ccb49b-f8fhw\" (UID: \"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:27.090779 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:27.090672    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:27.245265 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:27.245220    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"]
Jul 24 15:53:27.247252 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:53:27.247221    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1bb2616_d2d9_49d7_bab8_3ac80f3cea1e.slice/crio-54fb4e94fa83e80727fcc26005c393fb3486441e45efa8eac9e2ee83497a57f3 WatchSource:0}: Error finding container 54fb4e94fa83e80727fcc26005c393fb3486441e45efa8eac9e2ee83497a57f3: Status 404 returned error can't find the container with id 54fb4e94fa83e80727fcc26005c393fb3486441e45efa8eac9e2ee83497a57f3
Jul 24 15:53:28.152460 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:28.152417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw" event={"ID":"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e","Type":"ContainerStarted","Data":"54fb4e94fa83e80727fcc26005c393fb3486441e45efa8eac9e2ee83497a57f3"}
Jul 24 15:53:28.407916 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:28.407783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:28.546625 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:28.546573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:28.891294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:28.891261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:28.990876 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:28.990814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:29.143109 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:29.143023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:29.303584 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:29.303470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:29.404555 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:29.404396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:29.639675 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:29.639600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:29.737690 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:29.737545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:29.812141 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:29.812105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:30.052281 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:30.052243    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:53:30.052384 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:30.052311    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:53:30.164516 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:30.164481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw" event={"ID":"f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e","Type":"ContainerStarted","Data":"7f9043e34741dad09ef9ac46f04e16588985cbcaac49e102d8b5fa0272df8444"}
Jul 24 15:53:30.164730 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:30.164547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:53:30.206721 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:30.206627    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw" podStartSLOduration=1.40423713 podStartE2EDuration="4.206570892s" totalImagesPullingTime="2.802333762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:53:30.203457176 +0000 UTC m=+392.531021731" watchObservedRunningTime="2026-07-24 15:53:30.206570892 +0000 UTC m=+392.534135431"
Jul 24 15:53:31.171487 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:53:31.171459    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-f8fhw"
Jul 24 15:54:07.252466 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:07.252423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:09.087768 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.087733    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c645cc5b8-2w6j4"]
Jul 24 15:54:09.097319 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.097293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.101242 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.101215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c645cc5b8-2w6j4"]
Jul 24 15:54:09.136185 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.136147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:09.167933 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.167899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-trusted-ca-bundle\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.168120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.167952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d3269b8-8c39-4d05-8987-d199e1324436-console-serving-cert\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.168120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.167982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-oauth-serving-cert\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.168120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.168034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-service-ca\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.168120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.168061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-console-config\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.168120 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.168087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-768lg\" (UniqueName: \"kubernetes.io/projected/0d3269b8-8c39-4d05-8987-d199e1324436-kube-api-access-768lg\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.168322 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.168203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d3269b8-8c39-4d05-8987-d199e1324436-console-oauth-config\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.186716 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.186687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:09.270113 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-trusted-ca-bundle\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.270289 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d3269b8-8c39-4d05-8987-d199e1324436-console-serving-cert\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.270289 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-oauth-serving-cert\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.270289 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-service-ca\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.270289 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-console-config\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.270485 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-768lg\" (UniqueName: \"kubernetes.io/projected/0d3269b8-8c39-4d05-8987-d199e1324436-kube-api-access-768lg\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.270485 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d3269b8-8c39-4d05-8987-d199e1324436-console-oauth-config\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.271095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.270954    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-oauth-serving-cert\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.271095 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.271027    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-trusted-ca-bundle\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.271362 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.271319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-console-config\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.271419 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.271359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3269b8-8c39-4d05-8987-d199e1324436-service-ca\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.272836 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.272810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d3269b8-8c39-4d05-8987-d199e1324436-console-serving-cert\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.272980 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.272939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d3269b8-8c39-4d05-8987-d199e1324436-console-oauth-config\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.277240 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.277223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-768lg\" (UniqueName: \"kubernetes.io/projected/0d3269b8-8c39-4d05-8987-d199e1324436-kube-api-access-768lg\") pod \"console-5c645cc5b8-2w6j4\" (UID: \"0d3269b8-8c39-4d05-8987-d199e1324436\") " pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.408483 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.408398    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:09.535590 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:09.535566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c645cc5b8-2w6j4"]
Jul 24 15:54:09.537796 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:54:09.537768    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3269b8_8c39_4d05_8987_d199e1324436.slice/crio-15bf1dac98e76ea48b368431853f1b886aeffbb158fa83c17b853ac249c7bf5d WatchSource:0}: Error finding container 15bf1dac98e76ea48b368431853f1b886aeffbb158fa83c17b853ac249c7bf5d: Status 404 returned error can't find the container with id 15bf1dac98e76ea48b368431853f1b886aeffbb158fa83c17b853ac249c7bf5d
Jul 24 15:54:10.351568 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:10.351535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c645cc5b8-2w6j4" event={"ID":"0d3269b8-8c39-4d05-8987-d199e1324436","Type":"ContainerStarted","Data":"ba8eb633ee867e32028014e46e02a785c03fbfb6ce9aa75847ddfed250b20181"}
Jul 24 15:54:10.351939 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:10.351578    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c645cc5b8-2w6j4" event={"ID":"0d3269b8-8c39-4d05-8987-d199e1324436","Type":"ContainerStarted","Data":"15bf1dac98e76ea48b368431853f1b886aeffbb158fa83c17b853ac249c7bf5d"}
Jul 24 15:54:10.367643 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:10.367585    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c645cc5b8-2w6j4" podStartSLOduration=1.3675706220000001 podStartE2EDuration="1.367570622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:54:10.366635791 +0000 UTC m=+432.694200336" watchObservedRunningTime="2026-07-24 15:54:10.367570622 +0000 UTC m=+432.695135159"
Jul 24 15:54:18.640921 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:18.640887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:18.733352 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:18.733324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:18.846457 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:18.846419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:19.409252 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:19.409225    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:19.409413 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:19.409271    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:19.413987 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:19.413965    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:19.563178 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:19.563150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:19.760516 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:19.760444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:20.399892 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:20.399866    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c645cc5b8-2w6j4"
Jul 24 15:54:20.445069 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:20.445039    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8884465d-wkxv8"]
Jul 24 15:54:20.545535 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:20.545500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:20.646506 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:20.646472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:20.754779 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:20.754705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:33.972775 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:33.972739    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"]
Jul 24 15:54:33.976154 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:33.976138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:33.977926 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:33.977904    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mgtkk\""
Jul 24 15:54:33.978037 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:33.977905    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 15:54:33.978352 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:33.978339    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 15:54:33.983939 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:33.983917    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"]
Jul 24 15:54:34.068861 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.068838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.068964 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.068869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.069014 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.068985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrr9\" (UniqueName: \"kubernetes.io/projected/59c12460-bbc3-4f82-8724-5fc8804b2425-kube-api-access-qhrr9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.169413 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.169389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrr9\" (UniqueName: \"kubernetes.io/projected/59c12460-bbc3-4f82-8724-5fc8804b2425-kube-api-access-qhrr9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.169519 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.169430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.169519 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.169448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.169863 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.169844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.169863 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.169854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.177042 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.177020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrr9\" (UniqueName: \"kubernetes.io/projected/59c12460-bbc3-4f82-8724-5fc8804b2425-kube-api-access-qhrr9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.286463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.286442    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:34.366508 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.366357    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"]
Jul 24 15:54:34.370495 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.370472    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.378009 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.377951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"]
Jul 24 15:54:34.417772 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.417752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"]
Jul 24 15:54:34.419572 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:54:34.419544    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c12460_bbc3_4f82_8724_5fc8804b2425.slice/crio-61c387645476747efb75b11bad57f1bee2af70060c1a3b9877bbbd700d7fc741 WatchSource:0}: Error finding container 61c387645476747efb75b11bad57f1bee2af70060c1a3b9877bbbd700d7fc741: Status 404 returned error can't find the container with id 61c387645476747efb75b11bad57f1bee2af70060c1a3b9877bbbd700d7fc741
Jul 24 15:54:34.460460 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.460438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48" event={"ID":"59c12460-bbc3-4f82-8724-5fc8804b2425","Type":"ContainerStarted","Data":"61c387645476747efb75b11bad57f1bee2af70060c1a3b9877bbbd700d7fc741"}
Jul 24 15:54:34.472293 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.472271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhflc\" (UniqueName: \"kubernetes.io/projected/481a9218-3188-404e-8ce8-39e0958151e9-kube-api-access-zhflc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.472412 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.472337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.472460 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.472423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.573685 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.573605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.573685 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.573671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhflc\" (UniqueName: \"kubernetes.io/projected/481a9218-3188-404e-8ce8-39e0958151e9-kube-api-access-zhflc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.573829 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.573726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.573977 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.573957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.574012 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.573992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.580738 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.580719    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhflc\" (UniqueName: \"kubernetes.io/projected/481a9218-3188-404e-8ce8-39e0958151e9-kube-api-access-zhflc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.682597 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.682573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:34.805982 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.805958    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"]
Jul 24 15:54:34.808255 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:54:34.808227    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481a9218_3188_404e_8ce8_39e0958151e9.slice/crio-2bb53b48301a36cb375de1aeac3de345d5924cd315eb37ef123e4e0a58ac98f8 WatchSource:0}: Error finding container 2bb53b48301a36cb375de1aeac3de345d5924cd315eb37ef123e4e0a58ac98f8: Status 404 returned error can't find the container with id 2bb53b48301a36cb375de1aeac3de345d5924cd315eb37ef123e4e0a58ac98f8
Jul 24 15:54:34.973140 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.973113    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"]
Jul 24 15:54:34.976364 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.976349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:34.984257 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:34.984239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"]
Jul 24 15:54:35.077591 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.077523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.077591 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.077551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.077761 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.077647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5hg\" (UniqueName: \"kubernetes.io/projected/d939ea3b-bb02-475e-abd0-70d5f750bb56-kube-api-access-ch5hg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.178905 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.178873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5hg\" (UniqueName: \"kubernetes.io/projected/d939ea3b-bb02-475e-abd0-70d5f750bb56-kube-api-access-ch5hg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.178999 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.178940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.178999 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.178967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.179288 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.179271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.179373 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.179353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.186799 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.186778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5hg\" (UniqueName: \"kubernetes.io/projected/d939ea3b-bb02-475e-abd0-70d5f750bb56-kube-api-access-ch5hg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.286669 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.286640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:35.408854 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.408825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"]
Jul 24 15:54:35.411311 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:54:35.411280    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd939ea3b_bb02_475e_abd0_70d5f750bb56.slice/crio-999eacfa1120496a7b7e0bac4286f5aa216368e2655f0ceb2fa58e9890999083 WatchSource:0}: Error finding container 999eacfa1120496a7b7e0bac4286f5aa216368e2655f0ceb2fa58e9890999083: Status 404 returned error can't find the container with id 999eacfa1120496a7b7e0bac4286f5aa216368e2655f0ceb2fa58e9890999083
Jul 24 15:54:35.466539 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.466514    2577 generic.go:356] "Generic (PLEG): container finished" podID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerID="d8da34fb382f87d3b4af290f05cc3f4ce18be26c73bb7bb3b3c75c5c964aafdc" exitCode=0
Jul 24 15:54:35.466669 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.466592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48" event={"ID":"59c12460-bbc3-4f82-8724-5fc8804b2425","Type":"ContainerDied","Data":"d8da34fb382f87d3b4af290f05cc3f4ce18be26c73bb7bb3b3c75c5c964aafdc"}
Jul 24 15:54:35.468227 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.468205    2577 generic.go:356] "Generic (PLEG): container finished" podID="481a9218-3188-404e-8ce8-39e0958151e9" containerID="cb2d8c7fcc9843ba2ad9c0a9c5cdaef3baf5394006b77b3c67e6713cd88b14ad" exitCode=0
Jul 24 15:54:35.468324 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.468278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4" event={"ID":"481a9218-3188-404e-8ce8-39e0958151e9","Type":"ContainerDied","Data":"cb2d8c7fcc9843ba2ad9c0a9c5cdaef3baf5394006b77b3c67e6713cd88b14ad"}
Jul 24 15:54:35.468324 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.468313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4" event={"ID":"481a9218-3188-404e-8ce8-39e0958151e9","Type":"ContainerStarted","Data":"2bb53b48301a36cb375de1aeac3de345d5924cd315eb37ef123e4e0a58ac98f8"}
Jul 24 15:54:35.469586 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.469567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg" event={"ID":"d939ea3b-bb02-475e-abd0-70d5f750bb56","Type":"ContainerStarted","Data":"999eacfa1120496a7b7e0bac4286f5aa216368e2655f0ceb2fa58e9890999083"}
Jul 24 15:54:35.567713 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.567683    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"]
Jul 24 15:54:35.571140 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.571126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.578380 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.578356    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"]
Jul 24 15:54:35.682842 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.682769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc4dv\" (UniqueName: \"kubernetes.io/projected/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-kube-api-access-hc4dv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.682842 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.682831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.682973 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.682876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.784011 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.783978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc4dv\" (UniqueName: \"kubernetes.io/projected/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-kube-api-access-hc4dv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.784144 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.784032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.784144 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.784076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.784395 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.784380    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.784432 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.784417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.791054 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.791026    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc4dv\" (UniqueName: \"kubernetes.io/projected/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-kube-api-access-hc4dv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:35.881405 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:35.881381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:36.010492 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.010467    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"]
Jul 24 15:54:36.011959 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:54:36.011931    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ea0ec8_3bac_4d4e_8453_206ac8fe1209.slice/crio-8bd90a14e4e25f730c8a3fe60f9c7877ac04ea1c2785e5c7cb60c963831102ad WatchSource:0}: Error finding container 8bd90a14e4e25f730c8a3fe60f9c7877ac04ea1c2785e5c7cb60c963831102ad: Status 404 returned error can't find the container with id 8bd90a14e4e25f730c8a3fe60f9c7877ac04ea1c2785e5c7cb60c963831102ad
Jul 24 15:54:36.475577 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.475547    2577 generic.go:356] "Generic (PLEG): container finished" podID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerID="d66ddbdc8e9a8610160590f1fc6b52135ab9c63f59bf071ca3a93aa874b0c0b3" exitCode=0
Jul 24 15:54:36.475764 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.475648    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg" event={"ID":"d939ea3b-bb02-475e-abd0-70d5f750bb56","Type":"ContainerDied","Data":"d66ddbdc8e9a8610160590f1fc6b52135ab9c63f59bf071ca3a93aa874b0c0b3"}
Jul 24 15:54:36.477637 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.477579    2577 generic.go:356] "Generic (PLEG): container finished" podID="481a9218-3188-404e-8ce8-39e0958151e9" containerID="bb5b88e522fbe23ffb5aef597c6a4d67ed8cb02785a5953e139257833d859d53" exitCode=0
Jul 24 15:54:36.477720 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.477652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4" event={"ID":"481a9218-3188-404e-8ce8-39e0958151e9","Type":"ContainerDied","Data":"bb5b88e522fbe23ffb5aef597c6a4d67ed8cb02785a5953e139257833d859d53"}
Jul 24 15:54:36.479214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.479121    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerID="1930a446e14bc15a4bf5be894ab24464ad2455e8544c4136dcbc32e73d5ecdbb" exitCode=0
Jul 24 15:54:36.479214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.479152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m" event={"ID":"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209","Type":"ContainerDied","Data":"1930a446e14bc15a4bf5be894ab24464ad2455e8544c4136dcbc32e73d5ecdbb"}
Jul 24 15:54:36.479214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:36.479182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m" event={"ID":"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209","Type":"ContainerStarted","Data":"8bd90a14e4e25f730c8a3fe60f9c7877ac04ea1c2785e5c7cb60c963831102ad"}
Jul 24 15:54:37.485329 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.485297    2577 generic.go:356] "Generic (PLEG): container finished" podID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerID="fedceba949c81fb1317bf86f79f7c63d79bb788ca79229e76e14cbdfa44938b7" exitCode=0
Jul 24 15:54:37.485762 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.485357    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48" event={"ID":"59c12460-bbc3-4f82-8724-5fc8804b2425","Type":"ContainerDied","Data":"fedceba949c81fb1317bf86f79f7c63d79bb788ca79229e76e14cbdfa44938b7"}
Jul 24 15:54:37.487549 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.487528    2577 generic.go:356] "Generic (PLEG): container finished" podID="481a9218-3188-404e-8ce8-39e0958151e9" containerID="9323f9534d1e128f656a3d5c3034885f76145f777d15290666dc86b158cf0295" exitCode=0
Jul 24 15:54:37.487660 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.487596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4" event={"ID":"481a9218-3188-404e-8ce8-39e0958151e9","Type":"ContainerDied","Data":"9323f9534d1e128f656a3d5c3034885f76145f777d15290666dc86b158cf0295"}
Jul 24 15:54:37.489409 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.489391    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerID="9040768eec867009ca5092231e8c9136eef5e0a4ad12a11bc8647970e5026634" exitCode=0
Jul 24 15:54:37.489505 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.489471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m" event={"ID":"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209","Type":"ContainerDied","Data":"9040768eec867009ca5092231e8c9136eef5e0a4ad12a11bc8647970e5026634"}
Jul 24 15:54:37.491369 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.491348    2577 generic.go:356] "Generic (PLEG): container finished" podID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerID="9da30756aaef1c78c17ddcdf1255a676e6b39a8dcf007d13a11163b4f35b2361" exitCode=0
Jul 24 15:54:37.491465 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:37.491385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg" event={"ID":"d939ea3b-bb02-475e-abd0-70d5f750bb56","Type":"ContainerDied","Data":"9da30756aaef1c78c17ddcdf1255a676e6b39a8dcf007d13a11163b4f35b2361"}
Jul 24 15:54:38.498140 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.498107    2577 generic.go:356] "Generic (PLEG): container finished" podID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerID="6ee442937027ecca8f38c1457db290fbb08df40f4f146356d07586bd44f069fe" exitCode=0
Jul 24 15:54:38.498690 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.498190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg" event={"ID":"d939ea3b-bb02-475e-abd0-70d5f750bb56","Type":"ContainerDied","Data":"6ee442937027ecca8f38c1457db290fbb08df40f4f146356d07586bd44f069fe"}
Jul 24 15:54:38.500292 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.500273    2577 generic.go:356] "Generic (PLEG): container finished" podID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerID="61ea757cc2c8c1451d679266b602e3771c3ea1002ed3886556a947806fe0d56e" exitCode=0
Jul 24 15:54:38.500391 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.500337    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48" event={"ID":"59c12460-bbc3-4f82-8724-5fc8804b2425","Type":"ContainerDied","Data":"61ea757cc2c8c1451d679266b602e3771c3ea1002ed3886556a947806fe0d56e"}
Jul 24 15:54:38.502393 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.502370    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerID="93e72be9e9e6ef8d8763e600ab35a8c3a5efe02115380ab6ae546fd425e8bc8d" exitCode=0
Jul 24 15:54:38.502488 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.502408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m" event={"ID":"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209","Type":"ContainerDied","Data":"93e72be9e9e6ef8d8763e600ab35a8c3a5efe02115380ab6ae546fd425e8bc8d"}
Jul 24 15:54:38.635932 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.635910    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:38.711096 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.711064    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-util\") pod \"481a9218-3188-404e-8ce8-39e0958151e9\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") "
Jul 24 15:54:38.711294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.711132    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhflc\" (UniqueName: \"kubernetes.io/projected/481a9218-3188-404e-8ce8-39e0958151e9-kube-api-access-zhflc\") pod \"481a9218-3188-404e-8ce8-39e0958151e9\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") "
Jul 24 15:54:38.711294 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.711166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-bundle\") pod \"481a9218-3188-404e-8ce8-39e0958151e9\" (UID: \"481a9218-3188-404e-8ce8-39e0958151e9\") "
Jul 24 15:54:38.711845 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.711800    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-bundle" (OuterVolumeSpecName: "bundle") pod "481a9218-3188-404e-8ce8-39e0958151e9" (UID: "481a9218-3188-404e-8ce8-39e0958151e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:38.713518 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.713491    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481a9218-3188-404e-8ce8-39e0958151e9-kube-api-access-zhflc" (OuterVolumeSpecName: "kube-api-access-zhflc") pod "481a9218-3188-404e-8ce8-39e0958151e9" (UID: "481a9218-3188-404e-8ce8-39e0958151e9"). InnerVolumeSpecName "kube-api-access-zhflc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:54:38.716511 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.716473    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-util" (OuterVolumeSpecName: "util") pod "481a9218-3188-404e-8ce8-39e0958151e9" (UID: "481a9218-3188-404e-8ce8-39e0958151e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:38.812531 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.812489    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhflc\" (UniqueName: \"kubernetes.io/projected/481a9218-3188-404e-8ce8-39e0958151e9-kube-api-access-zhflc\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:38.812531 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.812529    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:38.812531 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:38.812540    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/481a9218-3188-404e-8ce8-39e0958151e9-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.512931 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.512897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4" event={"ID":"481a9218-3188-404e-8ce8-39e0958151e9","Type":"ContainerDied","Data":"2bb53b48301a36cb375de1aeac3de345d5924cd315eb37ef123e4e0a58ac98f8"}
Jul 24 15:54:39.512931 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.512926    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4"
Jul 24 15:54:39.513355 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.512950    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bb53b48301a36cb375de1aeac3de345d5924cd315eb37ef123e4e0a58ac98f8"
Jul 24 15:54:39.654092 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.654067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:39.702109 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.702087    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:39.704898 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.704882    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:39.719362 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.719322    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-util\") pod \"d939ea3b-bb02-475e-abd0-70d5f750bb56\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") "
Jul 24 15:54:39.719468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.719374    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch5hg\" (UniqueName: \"kubernetes.io/projected/d939ea3b-bb02-475e-abd0-70d5f750bb56-kube-api-access-ch5hg\") pod \"d939ea3b-bb02-475e-abd0-70d5f750bb56\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") "
Jul 24 15:54:39.719468 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.719403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-bundle\") pod \"d939ea3b-bb02-475e-abd0-70d5f750bb56\" (UID: \"d939ea3b-bb02-475e-abd0-70d5f750bb56\") "
Jul 24 15:54:39.719983 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.719958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-bundle" (OuterVolumeSpecName: "bundle") pod "d939ea3b-bb02-475e-abd0-70d5f750bb56" (UID: "d939ea3b-bb02-475e-abd0-70d5f750bb56"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:39.722017 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.721990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d939ea3b-bb02-475e-abd0-70d5f750bb56-kube-api-access-ch5hg" (OuterVolumeSpecName: "kube-api-access-ch5hg") pod "d939ea3b-bb02-475e-abd0-70d5f750bb56" (UID: "d939ea3b-bb02-475e-abd0-70d5f750bb56"). InnerVolumeSpecName "kube-api-access-ch5hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:54:39.725780 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.725747    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-util" (OuterVolumeSpecName: "util") pod "d939ea3b-bb02-475e-abd0-70d5f750bb56" (UID: "d939ea3b-bb02-475e-abd0-70d5f750bb56"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:39.820923 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.820897    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhrr9\" (UniqueName: \"kubernetes.io/projected/59c12460-bbc3-4f82-8724-5fc8804b2425-kube-api-access-qhrr9\") pod \"59c12460-bbc3-4f82-8724-5fc8804b2425\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") "
Jul 24 15:54:39.821094 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.820939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-bundle\") pod \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") "
Jul 24 15:54:39.821094 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.820980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-util\") pod \"59c12460-bbc3-4f82-8724-5fc8804b2425\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") "
Jul 24 15:54:39.821094 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc4dv\" (UniqueName: \"kubernetes.io/projected/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-kube-api-access-hc4dv\") pod \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") "
Jul 24 15:54:39.821094 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821054    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-bundle\") pod \"59c12460-bbc3-4f82-8724-5fc8804b2425\" (UID: \"59c12460-bbc3-4f82-8724-5fc8804b2425\") "
Jul 24 15:54:39.821288 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821146    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-util\") pod \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\" (UID: \"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209\") "
Jul 24 15:54:39.821471 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821450    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.821587 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821473    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch5hg\" (UniqueName: \"kubernetes.io/projected/d939ea3b-bb02-475e-abd0-70d5f750bb56-kube-api-access-ch5hg\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.821587 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821488    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d939ea3b-bb02-475e-abd0-70d5f750bb56-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.821587 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-bundle" (OuterVolumeSpecName: "bundle") pod "59c12460-bbc3-4f82-8724-5fc8804b2425" (UID: "59c12460-bbc3-4f82-8724-5fc8804b2425"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:39.821752 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.821610    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-bundle" (OuterVolumeSpecName: "bundle") pod "c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" (UID: "c3ea0ec8-3bac-4d4e-8453-206ac8fe1209"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:39.823046 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.823029    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c12460-bbc3-4f82-8724-5fc8804b2425-kube-api-access-qhrr9" (OuterVolumeSpecName: "kube-api-access-qhrr9") pod "59c12460-bbc3-4f82-8724-5fc8804b2425" (UID: "59c12460-bbc3-4f82-8724-5fc8804b2425"). InnerVolumeSpecName "kube-api-access-qhrr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:54:39.823436 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.823417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-kube-api-access-hc4dv" (OuterVolumeSpecName: "kube-api-access-hc4dv") pod "c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" (UID: "c3ea0ec8-3bac-4d4e-8453-206ac8fe1209"). InnerVolumeSpecName "kube-api-access-hc4dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:54:39.826681 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.826656    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-util" (OuterVolumeSpecName: "util") pod "c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" (UID: "c3ea0ec8-3bac-4d4e-8453-206ac8fe1209"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:39.853868 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.853835    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-util" (OuterVolumeSpecName: "util") pod "59c12460-bbc3-4f82-8724-5fc8804b2425" (UID: "59c12460-bbc3-4f82-8724-5fc8804b2425"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:39.922451 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.922426    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc4dv\" (UniqueName: \"kubernetes.io/projected/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-kube-api-access-hc4dv\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.922451 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.922449    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.922568 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.922459    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.922568 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.922468    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhrr9\" (UniqueName: \"kubernetes.io/projected/59c12460-bbc3-4f82-8724-5fc8804b2425-kube-api-access-qhrr9\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.922568 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.922477    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3ea0ec8-3bac-4d4e-8453-206ac8fe1209-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:39.922568 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:39.922484    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59c12460-bbc3-4f82-8724-5fc8804b2425-util\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:40.519734 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.519709    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48"
Jul 24 15:54:40.520127 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.519702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48" event={"ID":"59c12460-bbc3-4f82-8724-5fc8804b2425","Type":"ContainerDied","Data":"61c387645476747efb75b11bad57f1bee2af70060c1a3b9877bbbd700d7fc741"}
Jul 24 15:54:40.520127 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.519871    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61c387645476747efb75b11bad57f1bee2af70060c1a3b9877bbbd700d7fc741"
Jul 24 15:54:40.521769 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.521745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m" event={"ID":"c3ea0ec8-3bac-4d4e-8453-206ac8fe1209","Type":"ContainerDied","Data":"8bd90a14e4e25f730c8a3fe60f9c7877ac04ea1c2785e5c7cb60c963831102ad"}
Jul 24 15:54:40.521959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.521793    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bd90a14e4e25f730c8a3fe60f9c7877ac04ea1c2785e5c7cb60c963831102ad"
Jul 24 15:54:40.521959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.521750    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m"
Jul 24 15:54:40.523973 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.523955    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg"
Jul 24 15:54:40.524074 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.523976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg" event={"ID":"d939ea3b-bb02-475e-abd0-70d5f750bb56","Type":"ContainerDied","Data":"999eacfa1120496a7b7e0bac4286f5aa216368e2655f0ceb2fa58e9890999083"}
Jul 24 15:54:40.524074 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:40.524010    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="999eacfa1120496a7b7e0bac4286f5aa216368e2655f0ceb2fa58e9890999083"
Jul 24 15:54:42.016169 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:42.016133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:45.234139 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.234100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:45.468480 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.468445    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c8884465d-wkxv8" podUID="6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" containerName="console" containerID="cri-o://b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80" gracePeriod=15
Jul 24 15:54:45.716563 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.716539    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:54:45.868214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868143    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-serving-cert\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868214 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-service-ca\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czfqd\" (UniqueName: \"kubernetes.io/projected/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-kube-api-access-czfqd\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868247    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-config\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-oauth-config\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868320    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-oauth-serving-cert\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868444 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-trusted-ca-bundle\") pod \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\" (UID: \"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42\") "
Jul 24 15:54:45.868766 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868743    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-config" (OuterVolumeSpecName: "console-config") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:54:45.868838 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:54:45.868838 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868828    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:54:45.868960 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.868861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:54:45.870509 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.870482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:54:45.870594 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.870504    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:54:45.870594 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.870516    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-kube-api-access-czfqd" (OuterVolumeSpecName: "kube-api-access-czfqd") pod "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" (UID: "6b336f4c-9c24-42d6-8ef2-39fd2dd39d42"). InnerVolumeSpecName "kube-api-access-czfqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:54:45.968928 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968905    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:45.968928 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968926    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:45.969068 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968935    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czfqd\" (UniqueName: \"kubernetes.io/projected/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-kube-api-access-czfqd\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:45.969068 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968945    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:45.969068 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968959    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-console-oauth-config\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:45.969068 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968968    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-oauth-serving-cert\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:45.969068 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:45.968976    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42-trusted-ca-bundle\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 15:54:46.556556 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.556521    2577 generic.go:356] "Generic (PLEG): container finished" podID="6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" containerID="b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80" exitCode=0
Jul 24 15:54:46.557137 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.556638    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8884465d-wkxv8"
Jul 24 15:54:46.557137 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.556647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8884465d-wkxv8" event={"ID":"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42","Type":"ContainerDied","Data":"b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80"}
Jul 24 15:54:46.557137 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.556694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8884465d-wkxv8" event={"ID":"6b336f4c-9c24-42d6-8ef2-39fd2dd39d42","Type":"ContainerDied","Data":"72282dd85a378dc77cec6a46d33be98b6d69fdda2a89776036004a62f12820cf"}
Jul 24 15:54:46.557137 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.556721    2577 scope.go:117] "RemoveContainer" containerID="b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80"
Jul 24 15:54:46.566483 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.566462    2577 scope.go:117] "RemoveContainer" containerID="b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80"
Jul 24 15:54:46.566880 ip-10-0-133-118 kubenswrapper[2577]: E0724 15:54:46.566854    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80\": container with ID starting with b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80 not found: ID does not exist" containerID="b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80"
Jul 24 15:54:46.566945 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.566884    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80"} err="failed to get container status \"b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80\": rpc error: code = NotFound desc = could not find container \"b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80\": container with ID starting with b4f5caf9c783596739e291025408c8a1eaf2f868d11ac6e530e2e3cb74fc3c80 not found: ID does not exist"
Jul 24 15:54:46.573922 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.573898    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8884465d-wkxv8"]
Jul 24 15:54:46.577903 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:46.577872    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c8884465d-wkxv8"]
Jul 24 15:54:48.237673 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:48.237643    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" path="/var/lib/kubelet/pods/6b336f4c-9c24-42d6-8ef2-39fd2dd39d42/volumes"
Jul 24 15:54:48.669463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:48.669429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:49.817043 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:49.817002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:50.093960 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:50.093872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:50.628362 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:50.628329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:53.880883 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:53.880848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:54.098895 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:54.098858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:56.647608 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:56.647573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:57.318771 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:57.318734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:57.464371 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:57.464336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:58.050967 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:58.050938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:58.222516 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:54:58.222483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:01.640950 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:01.640909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:01.767442 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:01.767400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:10.892532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:10.892495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:10.990307 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:10.990269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:11.088057 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:11.088028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:13.855206 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:13.855158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:13.958967 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:13.958940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:16.346249 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:16.346198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:16.447281 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:16.447243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:19.699967 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:19.699931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:19.799941 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:19.799905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:21.331626 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:21.331592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:21.437469 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:21.437436    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:28.527702 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:28.527665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:28.628000 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:28.627970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:31.534409 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:31.534370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:31.637800 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:31.637764    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:34.148455 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:34.148420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:34.248690 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:34.248655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:37.747005 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:37.746973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:37.821028 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:37.820962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:37.874968 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:37.874931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:37.939365 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:37.939332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:39.501037 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.500987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:39.618385 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.618342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:39.670569 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.670538    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:39.887074 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887044    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:55:39.887346 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887335    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="pull"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887348    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="pull"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887358    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="util"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887363    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="util"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887370    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="pull"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887375    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="pull"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887381    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="extract"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887386    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="extract"
Jul 24 15:55:39.887392 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887393    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="pull"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887399    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="pull"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887406    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="pull"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887411    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="pull"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887421    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="util"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887426    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="util"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887433    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="util"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887437    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="util"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887445    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887450    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887457    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" containerName="console"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887462    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" containerName="console"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887469    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="util"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887474    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="util"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887481    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887486    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887496    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887501    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887553    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="481a9218-3188-404e-8ce8-39e0958151e9" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887560    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b336f4c-9c24-42d6-8ef2-39fd2dd39d42" containerName="console"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887568    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c12460-bbc3-4f82-8724-5fc8804b2425" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887574    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3ea0ec8-3bac-4d4e-8453-206ac8fe1209" containerName="extract"
Jul 24 15:55:39.887655 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.887580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d939ea3b-bb02-475e-abd0-70d5f750bb56" containerName="extract"
Jul 24 15:55:39.890461 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.890444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:39.892588 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.892567    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 15:55:39.892802 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.892573    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 15:55:39.892996 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.892980    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 15:55:39.892996 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.892990    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mgtkk\""
Jul 24 15:55:39.903604 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.903572    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:55:39.929299 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.929277    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:55:39.989837 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.989815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3a307db-b80c-4bff-95eb-0cfd6926b05e-config-file\") pod \"limitador-limitador-54d6d988d5-765wv\" (UID: \"f3a307db-b80c-4bff-95eb-0cfd6926b05e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:39.989959 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:39.989849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58gtg\" (UniqueName: \"kubernetes.io/projected/f3a307db-b80c-4bff-95eb-0cfd6926b05e-kube-api-access-58gtg\") pod \"limitador-limitador-54d6d988d5-765wv\" (UID: \"f3a307db-b80c-4bff-95eb-0cfd6926b05e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:40.091149 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.091127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3a307db-b80c-4bff-95eb-0cfd6926b05e-config-file\") pod \"limitador-limitador-54d6d988d5-765wv\" (UID: \"f3a307db-b80c-4bff-95eb-0cfd6926b05e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:40.091248 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.091160    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58gtg\" (UniqueName: \"kubernetes.io/projected/f3a307db-b80c-4bff-95eb-0cfd6926b05e-kube-api-access-58gtg\") pod \"limitador-limitador-54d6d988d5-765wv\" (UID: \"f3a307db-b80c-4bff-95eb-0cfd6926b05e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:40.091715 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.091698    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3a307db-b80c-4bff-95eb-0cfd6926b05e-config-file\") pod \"limitador-limitador-54d6d988d5-765wv\" (UID: \"f3a307db-b80c-4bff-95eb-0cfd6926b05e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:40.098264 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.098238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58gtg\" (UniqueName: \"kubernetes.io/projected/f3a307db-b80c-4bff-95eb-0cfd6926b05e-kube-api-access-58gtg\") pod \"limitador-limitador-54d6d988d5-765wv\" (UID: \"f3a307db-b80c-4bff-95eb-0cfd6926b05e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:40.202500 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.202450    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:40.333033 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.332963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:55:40.335076 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:55:40.335050    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a307db_b80c_4bff_95eb_0cfd6926b05e.slice/crio-c5d579710c20b01f595fbeaa761b628e6bb02a1d62280e02be4980484548cfef WatchSource:0}: Error finding container c5d579710c20b01f595fbeaa761b628e6bb02a1d62280e02be4980484548cfef: Status 404 returned error can't find the container with id c5d579710c20b01f595fbeaa761b628e6bb02a1d62280e02be4980484548cfef
Jul 24 15:55:40.343818 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.343787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:40.398620 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.398592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:40.828517 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:40.828473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv" event={"ID":"f3a307db-b80c-4bff-95eb-0cfd6926b05e","Type":"ContainerStarted","Data":"c5d579710c20b01f595fbeaa761b628e6bb02a1d62280e02be4980484548cfef"}
Jul 24 15:55:43.846740 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:43.846705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv" event={"ID":"f3a307db-b80c-4bff-95eb-0cfd6926b05e","Type":"ContainerStarted","Data":"9a7f38071b598ebe01169b89ede708574b9ed03d710d4bf22d4a5615670711cb"}
Jul 24 15:55:43.847180 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:43.846831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:55:43.861910 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:43.861868    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv" podStartSLOduration=2.332754548 podStartE2EDuration="4.861853256s" totalImagesPullingTime="2.529098708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:55:43.860700602 +0000 UTC m=+526.188265151" watchObservedRunningTime="2026-07-24 15:55:43.861853256 +0000 UTC m=+526.189417795"
Jul 24 15:55:49.697973 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:49.697937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:49.798805 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:49.798774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:50.134779 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:50.134739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:50.194059 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:50.194026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:54.335510 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:54.335460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:54.435592 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:54.435530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:54.852872 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:55:54.852844    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-765wv"
Jul 24 15:56:00.735967 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:00.735919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:00.835674 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:00.835640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:01.195592 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:01.195566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:10.344539 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:10.344490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:13.700662 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:13.700629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:14.015322 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:14.015198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:20.494494 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:20.494461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:20.561818 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:20.561781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:20.616488 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:20.616453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:20.956132 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:20.956077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:24.368591 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:24.368559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:24.467127 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:24.467091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:56.128304 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:56.128249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:56.226596 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:56.226563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:56.683789 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:56.683748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:57.187370 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:57.187336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:57.287631 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:57.287574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:57.344408 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:57.344368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:56:57.389960 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:57.389926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:58.170505 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:58.170463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:58.271276 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:56:58.271243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:07.608185 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:07.608140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:07.707318 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:07.707288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:26.817888 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:26.817795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:27.395697 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:27.395660    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:57:32.020791 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:32.020753    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:32.796846 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:32.796811    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:57:32.832792 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:32.832752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:37.494109 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:37.494023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:37.551777 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:37.551740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:38.515690 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:38.515651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:39.693945 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:39.693907    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:57:58.000527 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.000489    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"]
Jul 24 15:57:58.003985 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.003959    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.006642 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.006596    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zqxbv\""
Jul 24 15:57:58.006796 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.006688    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 15:57:58.006796 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.006711    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 15:57:58.006934 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.006814    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 15:57:58.014463 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.014430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"]
Jul 24 15:57:58.037311 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.037272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:58.098795 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.098753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.099013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.098843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/011d7423-df6a-49f1-beb2-5084e1fe4419-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.099013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.098882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjpml\" (UniqueName: \"kubernetes.io/projected/011d7423-df6a-49f1-beb2-5084e1fe4419-kube-api-access-kjpml\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.099013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.098918    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.099013 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.099001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.099218 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.099098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200336 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/011d7423-df6a-49f1-beb2-5084e1fe4419-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjpml\" (UniqueName: \"kubernetes.io/projected/011d7423-df6a-49f1-beb2-5084e1fe4419-kube-api-access-kjpml\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200525 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200762 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200820 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200793    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.200991 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.200971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.201142 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.201122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.202550 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.202531    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 15:57:58.202651 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.202576    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/011d7423-df6a-49f1-beb2-5084e1fe4419-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.207361 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.207335    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 15:57:58.213230 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.213202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/011d7423-df6a-49f1-beb2-5084e1fe4419-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.218133 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.218113    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 15:57:58.228890 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.228864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjpml\" (UniqueName: \"kubernetes.io/projected/011d7423-df6a-49f1-beb2-5084e1fe4419-kube-api-access-kjpml\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc\" (UID: \"011d7423-df6a-49f1-beb2-5084e1fe4419\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.319043 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.319016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zqxbv\""
Jul 24 15:57:58.327688 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.327659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:57:58.464532 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.464481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"]
Jul 24 15:57:58.467315 ip-10-0-133-118 kubenswrapper[2577]: W0724 15:57:58.467286    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011d7423_df6a_49f1_beb2_5084e1fe4419.slice/crio-481dee2d344d74f1176aad0c049e1cf549540de0b662ebdcbbc2f5f775df37a7 WatchSource:0}: Error finding container 481dee2d344d74f1176aad0c049e1cf549540de0b662ebdcbbc2f5f775df37a7: Status 404 returned error can't find the container with id 481dee2d344d74f1176aad0c049e1cf549540de0b662ebdcbbc2f5f775df37a7
Jul 24 15:57:58.468668 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.468651    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:57:58.515658 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:58.515625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc" event={"ID":"011d7423-df6a-49f1-beb2-5084e1fe4419","Type":"ContainerStarted","Data":"481dee2d344d74f1176aad0c049e1cf549540de0b662ebdcbbc2f5f775df37a7"}
Jul 24 15:57:59.688752 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:57:59.688713    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:58:00.528501 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:00.528465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc" event={"ID":"011d7423-df6a-49f1-beb2-5084e1fe4419","Type":"ContainerStarted","Data":"0d189a7674db56e47df599660e4b73986414d0509ed67eba5d89efb4c1d14ad3"}
Jul 24 15:58:00.545704 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:00.545646    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc" podStartSLOduration=2.352075977 podStartE2EDuration="3.54559737s" totalImagesPullingTime="1.193521393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:57:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:58:00.543776092 +0000 UTC m=+662.871340632" watchObservedRunningTime="2026-07-24 15:58:00.54559737 +0000 UTC m=+662.873161912"
Jul 24 15:58:08.328915 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:08.328877    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:58:08.329398 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:08.328943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:58:08.343044 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:08.343012    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:58:08.582239 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:08.582157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc"
Jul 24 15:58:09.985702 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:09.985660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:58:12.698119 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:12.698076    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:58:20.583085 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:20.583054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:58:24.593514 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:24.593482    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 15:58:37.235162 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:58:37.235121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:59:31.149430 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:59:31.149305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:59:31.224469 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:59:31.224434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:59:32.242560 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:59:32.242519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:59:32.355441 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:59:32.355400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 15:59:59.505669 ip-10-0-133-118 kubenswrapper[2577]: I0724 15:59:59.505631    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:00:00.151867 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:00.151836    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"]
Jul 24 16:00:00.155197 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:00.155181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"
Jul 24 16:00:00.157398 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:00.157376    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:00:00.163709 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:00.163683    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"]
Jul 24 16:00:00.165689 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:00.165664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"
Jul 24 16:00:00.308710 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:00.308680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"]
Jul 24 16:00:00.311651 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:00:00.311623    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc34d5d44_f892_4b8e_97a8_211d23284e88.slice/crio-95ff5a3d8add878d64f8fc766c9351b842ed19fd9fde4399693db88ddab57f25 WatchSource:0}: Error finding container 95ff5a3d8add878d64f8fc766c9351b842ed19fd9fde4399693db88ddab57f25: Status 404 returned error can't find the container with id 95ff5a3d8add878d64f8fc766c9351b842ed19fd9fde4399693db88ddab57f25
Jul 24 16:00:01.150320 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:01.150282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd" event={"ID":"c34d5d44-f892-4b8e-97a8-211d23284e88","Type":"ContainerStarted","Data":"95ff5a3d8add878d64f8fc766c9351b842ed19fd9fde4399693db88ddab57f25"}
Jul 24 16:00:03.162782 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:03.162745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd" event={"ID":"c34d5d44-f892-4b8e-97a8-211d23284e88","Type":"ContainerStarted","Data":"8e0cf2b79f861301b8c8fd57aeb3f3785aca8231f4dfafa1c66a2ebe16c44a06"}
Jul 24 16:00:03.180515 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:03.180450    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd" podStartSLOduration=0.832030498 podStartE2EDuration="3.180436205s" totalImagesPullingTime="2.348405707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:00:03.178264314 +0000 UTC m=+785.505828852" watchObservedRunningTime="2026-07-24 16:00:03.180436205 +0000 UTC m=+785.508000742"
Jul 24 16:00:04.169337 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:04.169299    2577 generic.go:356] "Generic (PLEG): container finished" podID="c34d5d44-f892-4b8e-97a8-211d23284e88" containerID="8e0cf2b79f861301b8c8fd57aeb3f3785aca8231f4dfafa1c66a2ebe16c44a06" exitCode=0
Jul 24 16:00:04.169731 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:04.169377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd" event={"ID":"c34d5d44-f892-4b8e-97a8-211d23284e88","Type":"ContainerDied","Data":"8e0cf2b79f861301b8c8fd57aeb3f3785aca8231f4dfafa1c66a2ebe16c44a06"}
Jul 24 16:00:05.310369 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:05.310347    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"
Jul 24 16:00:06.180932 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:06.180899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd" event={"ID":"c34d5d44-f892-4b8e-97a8-211d23284e88","Type":"ContainerDied","Data":"95ff5a3d8add878d64f8fc766c9351b842ed19fd9fde4399693db88ddab57f25"}
Jul 24 16:00:06.180932 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:06.180923    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-66xwd"
Jul 24 16:00:06.181146 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:06.180976    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95ff5a3d8add878d64f8fc766c9351b842ed19fd9fde4399693db88ddab57f25"
Jul 24 16:00:09.391195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:09.391166    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:00:20.314888 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:20.314794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:00:39.490305 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:39.490265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:00:39.588441 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:00:39.588399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:01:00.633973 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:01:00.633924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:01:36.449291 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:01:36.449246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:01:37.636162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:01:37.636128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:01.103493 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:01.103453    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:02:17.120458 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:17.120424    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:02:17.488925 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:17.488829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:17.663667 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:17.663623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:17.802284 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:17.802246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:23.884006 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:23.883970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:23.994817 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:23.994786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:24.119912 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:24.119879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:24.276604 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:24.276577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:28.894674 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:28.894634    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:02:30.004244 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:30.004209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:30.072696 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:30.072671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:48.998579 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:48.998544    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:02:56.586463 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:56.586427    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:02:58.296676 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:02:58.296639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:03:07.595038 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:03:07.595001    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:03:15.789400 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:03:15.789362    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:03:26.902945 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:03:26.902865    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:03:35.996356 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:03:35.996324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:03:47.589825 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:03:47.589795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:03:58.090319 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:03:58.090283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:04:19.695241 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:04:19.695207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:04:29.487661 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:04:29.487626    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:04:51.791571 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:04:51.791498    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:05:02.903528 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:02.903495    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:05:19.196441 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:19.196406    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:05:28.693348 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:28.693310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:05:38.901173 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:38.901139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:05:43.553271 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:43.553231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:05:48.891912 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:48.891880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:05:53.723491 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:53.723458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:05:58.784594 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:05:58.784563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:06:01.052412 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:06:01.052371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:07:46.900605 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:07:46.900573    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:07:56.810153 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:07:56.810082    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:08:10.002211 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:08:10.002175    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:08:21.597122 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:08:21.597087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:08:32.897404 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:08:32.897374    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:08:44.194980 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:08:44.194945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:08:53.191206 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:08:53.191173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:09:05.096012 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:09:05.095982    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:09:14.995458 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:09:14.995417    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:09:24.199543 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:09:24.199445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:09:37.997833 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:09:37.997798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:09:56.607914 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:09:56.607878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:10:06.396968 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:06.396933    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:10:07.782300 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.782261    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"]
Jul 24 16:10:07.782785 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.782769    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c34d5d44-f892-4b8e-97a8-211d23284e88" containerName="cleanup"
Jul 24 16:10:07.782834 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.782788    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34d5d44-f892-4b8e-97a8-211d23284e88" containerName="cleanup"
Jul 24 16:10:07.782874 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.782860    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c34d5d44-f892-4b8e-97a8-211d23284e88" containerName="cleanup"
Jul 24 16:10:07.786007 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.785988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.788411 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.788386    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-2d826dad-kserve-self-signed-certs\""
Jul 24 16:10:07.805240 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.805214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"]
Jul 24 16:10:07.823056 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.823033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tmp-dir\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.823172 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.823064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-dshm\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.823172 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.823082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24lbn\" (UniqueName: \"kubernetes.io/projected/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-kube-api-access-24lbn\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.823172 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.823102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-home\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.823280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.823216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-model-cache\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.823280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.823252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tls-certs\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.844577 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.844550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:07.924568 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.924542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24lbn\" (UniqueName: \"kubernetes.io/projected/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-kube-api-access-24lbn\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.924709 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.924576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-home\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.924709 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.924663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-model-cache\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.924709 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.924701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tls-certs\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.924877 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.924736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tmp-dir\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.924877 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.924765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-dshm\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.925096 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.925073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-model-cache\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.925167 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.925085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tmp-dir\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.925167 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.925123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-home\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.927076 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.927056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-dshm\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.927169 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.927152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tls-certs\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:07.932467 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:07.932444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24lbn\" (UniqueName: \"kubernetes.io/projected/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-kube-api-access-24lbn\") pod \"e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") " pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:08.095903 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:08.095811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:08.221105 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:08.221080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"]
Jul 24 16:10:08.222604 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:10:08.222580    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode36db1db_e99f_460c_8844_0c3fd9b3d1eb.slice/crio-3a6f5535fa77f380ee2933127a69ce9ad183af78a88412e5e061f2e7163987c2 WatchSource:0}: Error finding container 3a6f5535fa77f380ee2933127a69ce9ad183af78a88412e5e061f2e7163987c2: Status 404 returned error can't find the container with id 3a6f5535fa77f380ee2933127a69ce9ad183af78a88412e5e061f2e7163987c2
Jul 24 16:10:08.238307 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:08.238284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55" event={"ID":"e36db1db-e99f-460c-8844-0c3fd9b3d1eb","Type":"ContainerStarted","Data":"3a6f5535fa77f380ee2933127a69ce9ad183af78a88412e5e061f2e7163987c2"}
Jul 24 16:10:09.240059 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:09.240027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55" event={"ID":"e36db1db-e99f-460c-8844-0c3fd9b3d1eb","Type":"ContainerStarted","Data":"a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73"}
Jul 24 16:10:09.256973 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:09.256922    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55" podStartSLOduration=2.256908432 podStartE2EDuration="2.256908432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:10:09.255563666 +0000 UTC m=+1391.583128211" watchObservedRunningTime="2026-07-24 16:10:09.256908432 +0000 UTC m=+1391.584472969"
Jul 24 16:10:18.096679 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:18.096569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:18.096679 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:18.096643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:18.108954 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:18.108926    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:18.294180 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:18.294148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:10:20.622126 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:20.622090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:21.551772 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:21.551745    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:10:32.403247 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:32.403212    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:10:55.863719 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:55.863639    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"]
Jul 24 16:10:55.884056 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:55.884029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:57.436015 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:57.435979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:59.601777 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:10:59.601744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:11:10.882064 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:10.882026    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55" podUID="e36db1db-e99f-460c-8844-0c3fd9b3d1eb" containerName="main" containerID="cri-o://a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73" gracePeriod=45
Jul 24 16:11:11.124239 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.124216    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:11:11.240608 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240536    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tls-certs\") pod \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") "
Jul 24 16:11:11.240754 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240739    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tmp-dir\") pod \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") "
Jul 24 16:11:11.240828 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-dshm\") pod \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") "
Jul 24 16:11:11.240828 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240811    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-home\") pod \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") "
Jul 24 16:11:11.240940 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240836    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24lbn\" (UniqueName: \"kubernetes.io/projected/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-kube-api-access-24lbn\") pod \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") "
Jul 24 16:11:11.240940 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-model-cache\") pod \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\" (UID: \"e36db1db-e99f-460c-8844-0c3fd9b3d1eb\") "
Jul 24 16:11:11.241040 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.240937    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e36db1db-e99f-460c-8844-0c3fd9b3d1eb" (UID: "e36db1db-e99f-460c-8844-0c3fd9b3d1eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:11:11.241097 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.241055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-home" (OuterVolumeSpecName: "home") pod "e36db1db-e99f-460c-8844-0c3fd9b3d1eb" (UID: "e36db1db-e99f-460c-8844-0c3fd9b3d1eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:11:11.241170 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.241128    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:11:11.241225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.241169    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:11:11.241225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.241187    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "e36db1db-e99f-460c-8844-0c3fd9b3d1eb" (UID: "e36db1db-e99f-460c-8844-0c3fd9b3d1eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:11:11.242775 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.242755    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e36db1db-e99f-460c-8844-0c3fd9b3d1eb" (UID: "e36db1db-e99f-460c-8844-0c3fd9b3d1eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:11:11.242881 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.242775    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-dshm" (OuterVolumeSpecName: "dshm") pod "e36db1db-e99f-460c-8844-0c3fd9b3d1eb" (UID: "e36db1db-e99f-460c-8844-0c3fd9b3d1eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:11:11.242881 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.242780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-kube-api-access-24lbn" (OuterVolumeSpecName: "kube-api-access-24lbn") pod "e36db1db-e99f-460c-8844-0c3fd9b3d1eb" (UID: "e36db1db-e99f-460c-8844-0c3fd9b3d1eb"). InnerVolumeSpecName "kube-api-access-24lbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:11.342335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.342311    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:11:11.342335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.342331    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:11:11.342507 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.342344    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24lbn\" (UniqueName: \"kubernetes.io/projected/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-kube-api-access-24lbn\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:11:11.342507 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.342356    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36db1db-e99f-460c-8844-0c3fd9b3d1eb-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:11:11.551421 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.551393    2577 generic.go:356] "Generic (PLEG): container finished" podID="e36db1db-e99f-460c-8844-0c3fd9b3d1eb" containerID="a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73" exitCode=0
Jul 24 16:11:11.551523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.551437    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"
Jul 24 16:11:11.551523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.551437    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55" event={"ID":"e36db1db-e99f-460c-8844-0c3fd9b3d1eb","Type":"ContainerDied","Data":"a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73"}
Jul 24 16:11:11.551523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.551473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55" event={"ID":"e36db1db-e99f-460c-8844-0c3fd9b3d1eb","Type":"ContainerDied","Data":"3a6f5535fa77f380ee2933127a69ce9ad183af78a88412e5e061f2e7163987c2"}
Jul 24 16:11:11.551523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.551495    2577 scope.go:117] "RemoveContainer" containerID="a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73"
Jul 24 16:11:11.560601 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.560587    2577 scope.go:117] "RemoveContainer" containerID="a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73"
Jul 24 16:11:11.560890 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:11:11.560867    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73\": container with ID starting with a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73 not found: ID does not exist" containerID="a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73"
Jul 24 16:11:11.560959 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.560897    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73"} err="failed to get container status \"a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73\": rpc error: code = NotFound desc = could not find container \"a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73\": container with ID starting with a52f67191e7172eacb7bf986244fde743ef99cb4ebdce48e9902b305fce3ae73 not found: ID does not exist"
Jul 24 16:11:11.571211 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.571189    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"]
Jul 24 16:11:11.574630 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:11.574595    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-2d826dad-kserve-6cbfb5bc9f-jfk55"]
Jul 24 16:11:12.243165 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:12.243133    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36db1db-e99f-460c-8844-0c3fd9b3d1eb" path="/var/lib/kubelet/pods/e36db1db-e99f-460c-8844-0c3fd9b3d1eb/volumes"
Jul 24 16:11:14.091217 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:14.091184    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:11:21.596066 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:21.596033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:11:37.590382 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:37.590347    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:11:38.036959 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:38.036930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:51.801221 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:11:51.801185    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:12:01.393150 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:01.393113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:12:14.609272 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:14.609239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:12:24.087984 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:24.087908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:12:31.547019 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:31.546982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:37.497276 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:37.497236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:12:47.488722 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:47.488683    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:12:56.887254 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:56.887217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:56.954668 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:56.954641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:57.696378 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:12:57.696345    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:13:09.695247 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:13:09.695209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:13:21.289244 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:13:21.289212    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:13:29.795705 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:13:29.795674    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:13:41.097498 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:13:41.097465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:13:52.291723 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:13:52.291652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:14:04.385534 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:04.385505    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:14:12.798219 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:12.798182    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:14:15.926911 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:15.926871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:22.285627 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:22.285584    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:14:39.473194 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:39.473162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:42.478840 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:42.478800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:42.577053 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:42.577020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:42.942546 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:42.942510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:44.013085 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:44.013052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:14:48.652696 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:48.652667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:48.752856 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:48.752831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:48.900309 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:48.900283    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:49.187841 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:49.187811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:49.787252 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:49.787206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:51.483513 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:51.483480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:51.642703 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:51.642671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:51.818098 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:51.818045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:52.285134 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:52.285097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:52.384523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:52.384454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:57.896966 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:14:57.896934    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:15:00.130939 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.130908    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"]
Jul 24 16:15:00.131340 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.131249    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36db1db-e99f-460c-8844-0c3fd9b3d1eb" containerName="main"
Jul 24 16:15:00.131340 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.131260    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36db1db-e99f-460c-8844-0c3fd9b3d1eb" containerName="main"
Jul 24 16:15:00.131340 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.131318    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36db1db-e99f-460c-8844-0c3fd9b3d1eb" containerName="main"
Jul 24 16:15:00.134201 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.134179    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"]
Jul 24 16:15:00.134324 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.134292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"
Jul 24 16:15:00.136342 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.136322    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:15:00.137325 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.137310    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"
Jul 24 16:15:00.143975 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.143956    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"]
Jul 24 16:15:00.146354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.146335    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"]
Jul 24 16:15:00.146826 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.146809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"
Jul 24 16:15:00.154579 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.154563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"
Jul 24 16:15:00.308061 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.308038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"]
Jul 24 16:15:00.309794 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:15:00.309759    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3313ae2_3360_47cd_af88_9859558ea439.slice/crio-ba4389507bf9238e5028b53abadf4698b99b9998e70281db56ebec1cc9bff795 WatchSource:0}: Error finding container ba4389507bf9238e5028b53abadf4698b99b9998e70281db56ebec1cc9bff795: Status 404 returned error can't find the container with id ba4389507bf9238e5028b53abadf4698b99b9998e70281db56ebec1cc9bff795
Jul 24 16:15:00.493597 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.493572    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"]
Jul 24 16:15:00.495350 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:15:00.495326    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b3561d_27ab_4954_9576_fd1aa137f077.slice/crio-58f4b2e7d448f2deb4126682f6bc454243e91cbafb2ced496b6a016c5066872c WatchSource:0}: Error finding container 58f4b2e7d448f2deb4126682f6bc454243e91cbafb2ced496b6a016c5066872c: Status 404 returned error can't find the container with id 58f4b2e7d448f2deb4126682f6bc454243e91cbafb2ced496b6a016c5066872c
Jul 24 16:15:00.685015 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.684985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj" event={"ID":"c3313ae2-3360-47cd-af88-9859558ea439","Type":"ContainerStarted","Data":"7635e87568b8e2a78fe88a38e7f090a7619a12a0cd401a0c38852effde0c05bf"}
Jul 24 16:15:00.685295 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.685043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj" event={"ID":"c3313ae2-3360-47cd-af88-9859558ea439","Type":"ContainerStarted","Data":"ba4389507bf9238e5028b53abadf4698b99b9998e70281db56ebec1cc9bff795"}
Jul 24 16:15:00.686151 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.686132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx" event={"ID":"14b3561d-27ab-4954-9576-fd1aa137f077","Type":"ContainerStarted","Data":"58f4b2e7d448f2deb4126682f6bc454243e91cbafb2ced496b6a016c5066872c"}
Jul 24 16:15:00.700577 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:00.700539    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj" podStartSLOduration=0.700527369 podStartE2EDuration="700.527369ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:00.69831292 +0000 UTC m=+1683.025877480" watchObservedRunningTime="2026-07-24 16:15:00.700527369 +0000 UTC m=+1683.028091906"
Jul 24 16:15:01.692808 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:01.692778    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3313ae2-3360-47cd-af88-9859558ea439" containerID="7635e87568b8e2a78fe88a38e7f090a7619a12a0cd401a0c38852effde0c05bf" exitCode=0
Jul 24 16:15:01.693212 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:01.692863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj" event={"ID":"c3313ae2-3360-47cd-af88-9859558ea439","Type":"ContainerDied","Data":"7635e87568b8e2a78fe88a38e7f090a7619a12a0cd401a0c38852effde0c05bf"}
Jul 24 16:15:01.694272 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:01.694246    2577 generic.go:356] "Generic (PLEG): container finished" podID="14b3561d-27ab-4954-9576-fd1aa137f077" containerID="2aabd0b494429c91e3347e7d72876def80a7029dc10134e966c6e6e33156abd0" exitCode=0
Jul 24 16:15:01.694396 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:01.694322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx" event={"ID":"14b3561d-27ab-4954-9576-fd1aa137f077","Type":"ContainerDied","Data":"2aabd0b494429c91e3347e7d72876def80a7029dc10134e966c6e6e33156abd0"}
Jul 24 16:15:02.835573 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:02.835555    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"
Jul 24 16:15:02.853751 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:02.853726    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"
Jul 24 16:15:03.705203 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:03.705171    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"
Jul 24 16:15:03.705394 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:03.705208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj" event={"ID":"c3313ae2-3360-47cd-af88-9859558ea439","Type":"ContainerDied","Data":"ba4389507bf9238e5028b53abadf4698b99b9998e70281db56ebec1cc9bff795"}
Jul 24 16:15:03.705394 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:03.705255    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba4389507bf9238e5028b53abadf4698b99b9998e70281db56ebec1cc9bff795"
Jul 24 16:15:03.706523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:03.706498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx" event={"ID":"14b3561d-27ab-4954-9576-fd1aa137f077","Type":"ContainerDied","Data":"58f4b2e7d448f2deb4126682f6bc454243e91cbafb2ced496b6a016c5066872c"}
Jul 24 16:15:03.706523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:03.706514    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7hxdx"
Jul 24 16:15:03.706739 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:03.706534    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58f4b2e7d448f2deb4126682f6bc454243e91cbafb2ced496b6a016c5066872c"
Jul 24 16:15:17.033668 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.033639    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"]
Jul 24 16:15:17.034133 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.033950    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3313ae2-3360-47cd-af88-9859558ea439" containerName="cleanup"
Jul 24 16:15:17.034133 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.033961    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3313ae2-3360-47cd-af88-9859558ea439" containerName="cleanup"
Jul 24 16:15:17.034133 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.033973    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b3561d-27ab-4954-9576-fd1aa137f077" containerName="cleanup"
Jul 24 16:15:17.034133 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.033979    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b3561d-27ab-4954-9576-fd1aa137f077" containerName="cleanup"
Jul 24 16:15:17.034133 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.034041    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3313ae2-3360-47cd-af88-9859558ea439" containerName="cleanup"
Jul 24 16:15:17.034133 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.034048    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b3561d-27ab-4954-9576-fd1aa137f077" containerName="cleanup"
Jul 24 16:15:17.037199 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.037181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:17.039223 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.039206    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:15:17.039864 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.039843    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:15:17.049090 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.049068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"]
Jul 24 16:15:17.136008 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.135983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f250b0-b15a-47e1-8916-106b0eb32603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f991138c-7qfq8\" (UID: \"d9f250b0-b15a-47e1-8916-106b0eb32603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:17.236979 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.236958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f250b0-b15a-47e1-8916-106b0eb32603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f991138c-7qfq8\" (UID: \"d9f250b0-b15a-47e1-8916-106b0eb32603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:17.237483 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.237464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f250b0-b15a-47e1-8916-106b0eb32603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f991138c-7qfq8\" (UID: \"d9f250b0-b15a-47e1-8916-106b0eb32603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:17.346520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.346467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:17.676856 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.676834    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"]
Jul 24 16:15:17.678724 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:15:17.678701    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f250b0_b15a_47e1_8916_106b0eb32603.slice/crio-4ac4161b4ab9586b082936c2e5af9313501717e39cf55ac5cd365df5561a6ec9 WatchSource:0}: Error finding container 4ac4161b4ab9586b082936c2e5af9313501717e39cf55ac5cd365df5561a6ec9: Status 404 returned error can't find the container with id 4ac4161b4ab9586b082936c2e5af9313501717e39cf55ac5cd365df5561a6ec9
Jul 24 16:15:17.776867 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:17.776834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8" event={"ID":"d9f250b0-b15a-47e1-8916-106b0eb32603","Type":"ContainerStarted","Data":"4ac4161b4ab9586b082936c2e5af9313501717e39cf55ac5cd365df5561a6ec9"}
Jul 24 16:15:18.783047 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:18.783013    2577 generic.go:356] "Generic (PLEG): container finished" podID="d9f250b0-b15a-47e1-8916-106b0eb32603" containerID="bde5013d0f45fca848fe580e2ec4623fdd42887d2177cd81577bc656eed19b5a" exitCode=0
Jul 24 16:15:18.783400 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:18.783098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8" event={"ID":"d9f250b0-b15a-47e1-8916-106b0eb32603","Type":"ContainerDied","Data":"bde5013d0f45fca848fe580e2ec4623fdd42887d2177cd81577bc656eed19b5a"}
Jul 24 16:15:19.922876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:19.922847    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:20.059921 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:20.059824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f250b0-b15a-47e1-8916-106b0eb32603-maas-api-service-ca\") pod \"d9f250b0-b15a-47e1-8916-106b0eb32603\" (UID: \"d9f250b0-b15a-47e1-8916-106b0eb32603\") "
Jul 24 16:15:20.060174 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:20.060156    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9f250b0-b15a-47e1-8916-106b0eb32603-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9f250b0-b15a-47e1-8916-106b0eb32603" (UID: "d9f250b0-b15a-47e1-8916-106b0eb32603"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:15:20.160916 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:20.160880    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f250b0-b15a-47e1-8916-106b0eb32603-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:15:20.794847 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:20.794814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8" event={"ID":"d9f250b0-b15a-47e1-8916-106b0eb32603","Type":"ContainerDied","Data":"4ac4161b4ab9586b082936c2e5af9313501717e39cf55ac5cd365df5561a6ec9"}
Jul 24 16:15:20.795063 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:20.794897    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ac4161b4ab9586b082936c2e5af9313501717e39cf55ac5cd365df5561a6ec9"
Jul 24 16:15:20.795063 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:20.794828    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"
Jul 24 16:15:27.495700 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:27.495662    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"]
Jul 24 16:15:27.500223 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:27.500201    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-f991138c-29748495-qxmpj"]
Jul 24 16:15:27.503226 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:27.503207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:27.599107 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:27.599077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:27.699779 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:27.699088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:28.238656 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:28.238605    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3313ae2-3360-47cd-af88-9859558ea439" path="/var/lib/kubelet/pods/c3313ae2-3360-47cd-af88-9859558ea439/volumes"
Jul 24 16:15:30.596119 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:30.596087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:15:32.103729 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.103686    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"]
Jul 24 16:15:32.104352 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.104107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:32.106434 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.106411    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f991138c-7qfq8"]
Jul 24 16:15:32.237863 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.237837    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9f250b0-b15a-47e1-8916-106b0eb32603" path="/var/lib/kubelet/pods/d9f250b0-b15a-47e1-8916-106b0eb32603/volumes"
Jul 24 16:15:32.260854 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.260826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:32.503171 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.503089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:32.749705 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.749672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:32.845885 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.845859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:32.951135 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:32.951110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:33.259887 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:33.259814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:33.432238 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:33.432202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:35.867586 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:35.867554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:36.047205 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:36.047175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:36.148807 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:36.148736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:36.566920 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:36.566889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:36.678606 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:36.678574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:42.403014 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:15:42.402981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:16:00.805357 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.805316    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"]
Jul 24 16:16:00.805823 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.805688    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9f250b0-b15a-47e1-8916-106b0eb32603" containerName="revoke-keys"
Jul 24 16:16:00.805823 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.805703    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9f250b0-b15a-47e1-8916-106b0eb32603" containerName="revoke-keys"
Jul 24 16:16:00.805823 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.805788    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9f250b0-b15a-47e1-8916-106b0eb32603" containerName="revoke-keys"
Jul 24 16:16:00.808710 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.808691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:00.810547 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.810523    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:16:00.810693 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.810628    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:16:00.815647 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.815624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"]
Jul 24 16:16:00.871195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.871175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn\" (UID: \"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:00.972160 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.972127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn\" (UID: \"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:00.972750 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:00.972729    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn\" (UID: \"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:01.120448 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:01.120382    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:01.248023 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:01.247998    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"]
Jul 24 16:16:01.249888 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:16:01.249865    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0958ff8f_2fa4_4ec5_88f4_be30bd4197e0.slice/crio-c702486bcaee0e12a67e2951dd639669b93d9f8ae46fa75c08475b0db090df74 WatchSource:0}: Error finding container c702486bcaee0e12a67e2951dd639669b93d9f8ae46fa75c08475b0db090df74: Status 404 returned error can't find the container with id c702486bcaee0e12a67e2951dd639669b93d9f8ae46fa75c08475b0db090df74
Jul 24 16:16:02.006118 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:02.006091    2577 generic.go:356] "Generic (PLEG): container finished" podID="0958ff8f-2fa4-4ec5-88f4-be30bd4197e0" containerID="f382133e558dc33299b6b7a453f070c4c1b13a00e3c36828d4f2d81eedf5a3ae" exitCode=0
Jul 24 16:16:02.006424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:02.006165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn" event={"ID":"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0","Type":"ContainerDied","Data":"f382133e558dc33299b6b7a453f070c4c1b13a00e3c36828d4f2d81eedf5a3ae"}
Jul 24 16:16:02.006424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:02.006222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn" event={"ID":"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0","Type":"ContainerStarted","Data":"c702486bcaee0e12a67e2951dd639669b93d9f8ae46fa75c08475b0db090df74"}
Jul 24 16:16:03.143153 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:03.143133    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:03.189841 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:03.189817    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0-maas-api-service-ca\") pod \"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0\" (UID: \"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0\") "
Jul 24 16:16:03.190125 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:03.190107    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0958ff8f-2fa4-4ec5-88f4-be30bd4197e0" (UID: "0958ff8f-2fa4-4ec5-88f4-be30bd4197e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:16:03.290230 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:03.290212    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:16:04.017279 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:04.017251    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"
Jul 24 16:16:04.017462 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:04.017251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn" event={"ID":"0958ff8f-2fa4-4ec5-88f4-be30bd4197e0","Type":"ContainerDied","Data":"c702486bcaee0e12a67e2951dd639669b93d9f8ae46fa75c08475b0db090df74"}
Jul 24 16:16:04.017462 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:04.017407    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c702486bcaee0e12a67e2951dd639669b93d9f8ae46fa75c08475b0db090df74"
Jul 24 16:16:11.240238 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:11.240206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:13.207417 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:13.207382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:14.294504 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:14.294471    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:16:15.888521 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:15.888489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:15.889188 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:15.889123    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"]
Jul 24 16:16:15.891305 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:15.891282    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b92f583-xw8kn"]
Jul 24 16:16:16.051986 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:16.051931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:16.237933 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:16.237865    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0958ff8f-2fa4-4ec5-88f4-be30bd4197e0" path="/var/lib/kubelet/pods/0958ff8f-2fa4-4ec5-88f4-be30bd4197e0/volumes"
Jul 24 16:16:16.276876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:16.276852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:16.505635 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:16.505540    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:16.835268 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:16.835237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:17.195327 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:17.195220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:18.976012 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:18.975966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:19.075195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.075167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:19.217750 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.217719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:19.587489 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.587447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:19.821354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.821318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:19.918193 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.918116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:19.974890 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.974867    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"]
Jul 24 16:16:19.975180 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.975169    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0958ff8f-2fa4-4ec5-88f4-be30bd4197e0" containerName="revoke-keys"
Jul 24 16:16:19.975225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.975182    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0958ff8f-2fa4-4ec5-88f4-be30bd4197e0" containerName="revoke-keys"
Jul 24 16:16:19.975260 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.975245    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0958ff8f-2fa4-4ec5-88f4-be30bd4197e0" containerName="revoke-keys"
Jul 24 16:16:19.978591 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.978573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:19.980727 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.980710    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2cc8eb19\""
Jul 24 16:16:19.980831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.980712    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2cc8eb19-dockercfg-6glt7\""
Jul 24 16:16:19.990809 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:19.990788    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"]
Jul 24 16:16:20.019221 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.019197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ce6f37b-e658-4aee-8649-e327e59224af-plugins-config-volume\") pod \"payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") " pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.019346 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.019265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vcx6\" (UniqueName: \"kubernetes.io/projected/8ce6f37b-e658-4aee-8649-e327e59224af-kube-api-access-8vcx6\") pod \"payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") " pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.120162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.120134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ce6f37b-e658-4aee-8649-e327e59224af-plugins-config-volume\") pod \"payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") " pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.120304 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.120199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vcx6\" (UniqueName: \"kubernetes.io/projected/8ce6f37b-e658-4aee-8649-e327e59224af-kube-api-access-8vcx6\") pod \"payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") " pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.120906 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.120880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ce6f37b-e658-4aee-8649-e327e59224af-plugins-config-volume\") pod \"payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") " pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.128643 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.128606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vcx6\" (UniqueName: \"kubernetes.io/projected/8ce6f37b-e658-4aee-8649-e327e59224af-kube-api-access-8vcx6\") pod \"payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") " pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.287974 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.287948    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:20.620949 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.620925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"]
Jul 24 16:16:20.622561 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:16:20.622532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce6f37b_e658_4aee_8649_e327e59224af.slice/crio-3f675f268c73b210351e41a79a8c84f02ab51d8e94f0c1793ebc86a7373cd1f5 WatchSource:0}: Error finding container 3f675f268c73b210351e41a79a8c84f02ab51d8e94f0c1793ebc86a7373cd1f5: Status 404 returned error can't find the container with id 3f675f268c73b210351e41a79a8c84f02ab51d8e94f0c1793ebc86a7373cd1f5
Jul 24 16:16:20.624896 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:20.624872    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:16:21.115611 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:21.115578    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb" event={"ID":"8ce6f37b-e658-4aee-8649-e327e59224af","Type":"ContainerStarted","Data":"3f675f268c73b210351e41a79a8c84f02ab51d8e94f0c1793ebc86a7373cd1f5"}
Jul 24 16:16:23.128701 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:23.128668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb" event={"ID":"8ce6f37b-e658-4aee-8649-e327e59224af","Type":"ContainerStarted","Data":"fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc"}
Jul 24 16:16:23.129061 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:23.128775    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:23.147993 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:23.147947    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb" podStartSLOduration=2.3560055699999998 podStartE2EDuration="4.147935453s" totalImagesPullingTime="1.791929883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:16:23.145181604 +0000 UTC m=+1765.472746141" watchObservedRunningTime="2026-07-24 16:16:23.147935453 +0000 UTC m=+1765.475499990"
Jul 24 16:16:25.492408 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:25.492379    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:16:34.136565 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:34.136536    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:44.766833 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:44.766800    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"]
Jul 24 16:16:44.770913 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:44.770896    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:44.772947 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:44.772927    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:16:44.773048 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:44.772992    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:16:44.777271 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:44.777252    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"]
Jul 24 16:16:44.903537 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:44.903495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl\" (UID: \"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:45.004824 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:45.004789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl\" (UID: \"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:45.005384 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:45.005360    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl\" (UID: \"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:45.083672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:45.083653    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:45.206846 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:45.206823    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"]
Jul 24 16:16:45.208266 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:16:45.208240    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4af4ab_9f68_4f5a_a8b3_00c5879e2610.slice/crio-e73ca3152ce0b4727acbef279a28a4c35eea372993591f29b17fade684bffce0 WatchSource:0}: Error finding container e73ca3152ce0b4727acbef279a28a4c35eea372993591f29b17fade684bffce0: Status 404 returned error can't find the container with id e73ca3152ce0b4727acbef279a28a4c35eea372993591f29b17fade684bffce0
Jul 24 16:16:45.246774 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:45.246753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl" event={"ID":"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610","Type":"ContainerStarted","Data":"e73ca3152ce0b4727acbef279a28a4c35eea372993591f29b17fade684bffce0"}
Jul 24 16:16:46.254325 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:46.254258    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d4af4ab-9f68-4f5a-a8b3-00c5879e2610" containerID="5a5ba9490a46c2a038cae30cd39998fde5f8d01e137553e1aedbd616802fdc24" exitCode=0
Jul 24 16:16:46.254677 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:46.254339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl" event={"ID":"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610","Type":"ContainerDied","Data":"5a5ba9490a46c2a038cae30cd39998fde5f8d01e137553e1aedbd616802fdc24"}
Jul 24 16:16:47.389002 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:47.388982    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:47.524846 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:47.524820    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610-maas-api-service-ca\") pod \"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610\" (UID: \"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610\") "
Jul 24 16:16:47.525145 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:47.525126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d4af4ab-9f68-4f5a-a8b3-00c5879e2610" (UID: "6d4af4ab-9f68-4f5a-a8b3-00c5879e2610"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:16:47.626264 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:47.626243    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:16:48.265645 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:48.265602    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"
Jul 24 16:16:48.265772 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:48.265640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl" event={"ID":"6d4af4ab-9f68-4f5a-a8b3-00c5879e2610","Type":"ContainerDied","Data":"e73ca3152ce0b4727acbef279a28a4c35eea372993591f29b17fade684bffce0"}
Jul 24 16:16:48.265772 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:48.265684    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e73ca3152ce0b4727acbef279a28a4c35eea372993591f29b17fade684bffce0"
Jul 24 16:16:55.237484 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.237402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:55.264120 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.264090    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"]
Jul 24 16:16:55.264309 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.264284    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb" podUID="8ce6f37b-e658-4aee-8649-e327e59224af" containerName="payload-processing" containerID="cri-o://fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc" gracePeriod=30
Jul 24 16:16:55.304316 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.304292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:55.411859 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.406536    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:55.518589 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.518566    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:55.587151 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.587118    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ce6f37b-e658-4aee-8649-e327e59224af-plugins-config-volume\") pod \"8ce6f37b-e658-4aee-8649-e327e59224af\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") "
Jul 24 16:16:55.587336 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.587215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vcx6\" (UniqueName: \"kubernetes.io/projected/8ce6f37b-e658-4aee-8649-e327e59224af-kube-api-access-8vcx6\") pod \"8ce6f37b-e658-4aee-8649-e327e59224af\" (UID: \"8ce6f37b-e658-4aee-8649-e327e59224af\") "
Jul 24 16:16:55.587445 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.587422    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ce6f37b-e658-4aee-8649-e327e59224af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ce6f37b-e658-4aee-8649-e327e59224af" (UID: "8ce6f37b-e658-4aee-8649-e327e59224af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:16:55.589438 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.589408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce6f37b-e658-4aee-8649-e327e59224af-kube-api-access-8vcx6" (OuterVolumeSpecName: "kube-api-access-8vcx6") pod "8ce6f37b-e658-4aee-8649-e327e59224af" (UID: "8ce6f37b-e658-4aee-8649-e327e59224af"). InnerVolumeSpecName "kube-api-access-8vcx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:16:55.688066 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.688038    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ce6f37b-e658-4aee-8649-e327e59224af-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:16:55.688066 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:55.688061    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vcx6\" (UniqueName: \"kubernetes.io/projected/8ce6f37b-e658-4aee-8649-e327e59224af-kube-api-access-8vcx6\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:16:56.308944 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.308914    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ce6f37b-e658-4aee-8649-e327e59224af" containerID="fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc" exitCode=0
Jul 24 16:16:56.309288 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.308947    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"
Jul 24 16:16:56.309288 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.309003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb" event={"ID":"8ce6f37b-e658-4aee-8649-e327e59224af","Type":"ContainerDied","Data":"fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc"}
Jul 24 16:16:56.309288 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.309045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb" event={"ID":"8ce6f37b-e658-4aee-8649-e327e59224af","Type":"ContainerDied","Data":"3f675f268c73b210351e41a79a8c84f02ab51d8e94f0c1793ebc86a7373cd1f5"}
Jul 24 16:16:56.309288 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.309074    2577 scope.go:117] "RemoveContainer" containerID="fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc"
Jul 24 16:16:56.317788 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.317774    2577 scope.go:117] "RemoveContainer" containerID="fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc"
Jul 24 16:16:56.318061 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:16:56.318044    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc\": container with ID starting with fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc not found: ID does not exist" containerID="fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc"
Jul 24 16:16:56.318129 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.318066    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc"} err="failed to get container status \"fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc\": rpc error: code = NotFound desc = could not find container \"fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc\": container with ID starting with fb1993a0b9cd5f5e25db188680d7e87fd298718abfd06f601b6b873e5882c9bc not found: ID does not exist"
Jul 24 16:16:56.326201 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.326182    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"]
Jul 24 16:16:56.332455 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:56.332436    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2cc8eb19-854597cbbb-rg7kb"]
Jul 24 16:16:58.193419 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:58.193389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:16:58.238486 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:58.238458    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce6f37b-e658-4aee-8649-e327e59224af" path="/var/lib/kubelet/pods/8ce6f37b-e658-4aee-8649-e327e59224af/volumes"
Jul 24 16:16:59.845556 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:59.845525    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"]
Jul 24 16:16:59.849944 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:59.849914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:59.850525 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:16:59.850507    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cc8eb19-dtgpl"]
Jul 24 16:17:00.237569 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:00.237496    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d4af4ab-9f68-4f5a-a8b3-00c5879e2610" path="/var/lib/kubelet/pods/6d4af4ab-9f68-4f5a-a8b3-00c5879e2610/volumes"
Jul 24 16:17:00.997236 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:00.997210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:01.246336 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:01.246309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:01.470525 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:01.470495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:01.786402 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:01.786376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:02.249271 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:02.249185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:04.321374 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:04.321327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:04.404943 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:04.404877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:04.463716 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:04.463680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:04.994206 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:04.994170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:05.163369 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.163339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:05.252315 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.252247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:05.299563 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299535    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"]
Jul 24 16:17:05.299902 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4af4ab-9f68-4f5a-a8b3-00c5879e2610" containerName="revoke-keys"
Jul 24 16:17:05.299947 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299904    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4af4ab-9f68-4f5a-a8b3-00c5879e2610" containerName="revoke-keys"
Jul 24 16:17:05.299947 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299915    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce6f37b-e658-4aee-8649-e327e59224af" containerName="payload-processing"
Jul 24 16:17:05.299947 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299921    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce6f37b-e658-4aee-8649-e327e59224af" containerName="payload-processing"
Jul 24 16:17:05.300044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299975    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4af4ab-9f68-4f5a-a8b3-00c5879e2610" containerName="revoke-keys"
Jul 24 16:17:05.300044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.299983    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce6f37b-e658-4aee-8649-e327e59224af" containerName="payload-processing"
Jul 24 16:17:05.303109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.303094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.305140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.305122    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-8426fff8-dockercfg-nbqgq\""
Jul 24 16:17:05.305245 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.305162    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-8426fff8\""
Jul 24 16:17:05.313890 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.313870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"]
Jul 24 16:17:05.339640 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.339602    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"]
Jul 24 16:17:05.342774 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.342759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.351393 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.351373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"]
Jul 24 16:17:05.462025 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.461997    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b74deb-bb21-4665-9722-e39b4d0675b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.462191 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.462035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f121208-950f-4c5e-89e4-19e9c5473a0d-plugins-config-volume\") pod \"payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") " pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.462191 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.462144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzj5b\" (UniqueName: \"kubernetes.io/projected/f8b74deb-bb21-4665-9722-e39b4d0675b6-kube-api-access-rzj5b\") pod \"payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.462278 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.462191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6tfl\" (UniqueName: \"kubernetes.io/projected/6f121208-950f-4c5e-89e4-19e9c5473a0d-kube-api-access-h6tfl\") pod \"payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") " pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.562607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.562576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzj5b\" (UniqueName: \"kubernetes.io/projected/f8b74deb-bb21-4665-9722-e39b4d0675b6-kube-api-access-rzj5b\") pod \"payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.562780 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.562662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6tfl\" (UniqueName: \"kubernetes.io/projected/6f121208-950f-4c5e-89e4-19e9c5473a0d-kube-api-access-h6tfl\") pod \"payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") " pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.562780 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.562702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b74deb-bb21-4665-9722-e39b4d0675b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.562913 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.562847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f121208-950f-4c5e-89e4-19e9c5473a0d-plugins-config-volume\") pod \"payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") " pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.563493 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.563472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f121208-950f-4c5e-89e4-19e9c5473a0d-plugins-config-volume\") pod \"payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") " pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.563540 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.563480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b74deb-bb21-4665-9722-e39b4d0675b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.572285 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.572263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6tfl\" (UniqueName: \"kubernetes.io/projected/6f121208-950f-4c5e-89e4-19e9c5473a0d-kube-api-access-h6tfl\") pod \"payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") " pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.572741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.572721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzj5b\" (UniqueName: \"kubernetes.io/projected/f8b74deb-bb21-4665-9722-e39b4d0675b6-kube-api-access-rzj5b\") pod \"payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.613355 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.613327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:05.653321 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.653287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:05.757156 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.757118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"]
Jul 24 16:17:05.760395 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:17:05.760341    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f121208_950f_4c5e_89e4_19e9c5473a0d.slice/crio-4cbc964eb456dce5eed2a14af745156f90c0c404ab5a007a8d1fd14b40af52f6 WatchSource:0}: Error finding container 4cbc964eb456dce5eed2a14af745156f90c0c404ab5a007a8d1fd14b40af52f6: Status 404 returned error can't find the container with id 4cbc964eb456dce5eed2a14af745156f90c0c404ab5a007a8d1fd14b40af52f6
Jul 24 16:17:05.810051 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:05.810026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"]
Jul 24 16:17:05.811770 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:17:05.811746    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b74deb_bb21_4665_9722_e39b4d0675b6.slice/crio-e83d47bd3ee1127dc132e6781a323f7e15a40fe4c146dfe50c618a587b9b117c WatchSource:0}: Error finding container e83d47bd3ee1127dc132e6781a323f7e15a40fe4c146dfe50c618a587b9b117c: Status 404 returned error can't find the container with id e83d47bd3ee1127dc132e6781a323f7e15a40fe4c146dfe50c618a587b9b117c
Jul 24 16:17:06.363343 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.363312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz" event={"ID":"f8b74deb-bb21-4665-9722-e39b4d0675b6","Type":"ContainerStarted","Data":"9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96"}
Jul 24 16:17:06.363788 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.363358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz" event={"ID":"f8b74deb-bb21-4665-9722-e39b4d0675b6","Type":"ContainerStarted","Data":"e83d47bd3ee1127dc132e6781a323f7e15a40fe4c146dfe50c618a587b9b117c"}
Jul 24 16:17:06.363788 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.363390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:06.364897 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.364871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62" event={"ID":"6f121208-950f-4c5e-89e4-19e9c5473a0d","Type":"ContainerStarted","Data":"a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f"}
Jul 24 16:17:06.364991 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.364913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62" event={"ID":"6f121208-950f-4c5e-89e4-19e9c5473a0d","Type":"ContainerStarted","Data":"4cbc964eb456dce5eed2a14af745156f90c0c404ab5a007a8d1fd14b40af52f6"}
Jul 24 16:17:06.365056 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.365004    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:06.386031 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.385984    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz" podStartSLOduration=1.385968474 podStartE2EDuration="1.385968474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:06.383127468 +0000 UTC m=+1808.710692005" watchObservedRunningTime="2026-07-24 16:17:06.385968474 +0000 UTC m=+1808.713533013"
Jul 24 16:17:06.401197 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:06.401146    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62" podStartSLOduration=1.40113278 podStartE2EDuration="1.40113278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:06.400710633 +0000 UTC m=+1808.728275373" watchObservedRunningTime="2026-07-24 16:17:06.40113278 +0000 UTC m=+1808.728697319"
Jul 24 16:17:09.796683 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:09.796651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:17:17.372147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:17.372111    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:17.372581 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:17.372170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:29.070894 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.070864    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"]
Jul 24 16:17:29.075240 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.075224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:29.077091 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.077065    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:17:29.077348 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.077326    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:17:29.082886 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.082832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"]
Jul 24 16:17:29.148147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.148124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51a17c26-afbd-4568-a27d-0402e3821fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8426fff8-78gqn\" (UID: \"51a17c26-afbd-4568-a27d-0402e3821fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:29.249148 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.249120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51a17c26-afbd-4568-a27d-0402e3821fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8426fff8-78gqn\" (UID: \"51a17c26-afbd-4568-a27d-0402e3821fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:29.249776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.249754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51a17c26-afbd-4568-a27d-0402e3821fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8426fff8-78gqn\" (UID: \"51a17c26-afbd-4568-a27d-0402e3821fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:29.390470 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.390413    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:29.517271 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:29.517245    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"]
Jul 24 16:17:29.520518 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:17:29.520493    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a17c26_afbd_4568_a27d_0402e3821fc9.slice/crio-b7c3c5aa8f0d899d1d6302e9f353d7a475ddcde786e1ce9eb38848d8f6951d9d WatchSource:0}: Error finding container b7c3c5aa8f0d899d1d6302e9f353d7a475ddcde786e1ce9eb38848d8f6951d9d: Status 404 returned error can't find the container with id b7c3c5aa8f0d899d1d6302e9f353d7a475ddcde786e1ce9eb38848d8f6951d9d
Jul 24 16:17:30.495697 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:30.495602    2577 generic.go:356] "Generic (PLEG): container finished" podID="51a17c26-afbd-4568-a27d-0402e3821fc9" containerID="963b722fba14ac94bab3d3ed0cf3cba442fb16f0a3c98bdfbf40871f10d9cd46" exitCode=0
Jul 24 16:17:30.496014 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:30.495690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn" event={"ID":"51a17c26-afbd-4568-a27d-0402e3821fc9","Type":"ContainerDied","Data":"963b722fba14ac94bab3d3ed0cf3cba442fb16f0a3c98bdfbf40871f10d9cd46"}
Jul 24 16:17:30.496014 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:30.495749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn" event={"ID":"51a17c26-afbd-4568-a27d-0402e3821fc9","Type":"ContainerStarted","Data":"b7c3c5aa8f0d899d1d6302e9f353d7a475ddcde786e1ce9eb38848d8f6951d9d"}
Jul 24 16:17:31.630882 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:31.630859    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:31.665957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:31.665928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51a17c26-afbd-4568-a27d-0402e3821fc9-maas-api-service-ca\") pod \"51a17c26-afbd-4568-a27d-0402e3821fc9\" (UID: \"51a17c26-afbd-4568-a27d-0402e3821fc9\") "
Jul 24 16:17:31.666239 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:31.666219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51a17c26-afbd-4568-a27d-0402e3821fc9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51a17c26-afbd-4568-a27d-0402e3821fc9" (UID: "51a17c26-afbd-4568-a27d-0402e3821fc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:17:31.767231 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:31.767172    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51a17c26-afbd-4568-a27d-0402e3821fc9-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:17:32.507074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:32.507008    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"
Jul 24 16:17:32.507074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:32.507050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn" event={"ID":"51a17c26-afbd-4568-a27d-0402e3821fc9","Type":"ContainerDied","Data":"b7c3c5aa8f0d899d1d6302e9f353d7a475ddcde786e1ce9eb38848d8f6951d9d"}
Jul 24 16:17:32.507289 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:32.507093    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7c3c5aa8f0d899d1d6302e9f353d7a475ddcde786e1ce9eb38848d8f6951d9d"
Jul 24 16:17:39.672996 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.672960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:39.696340 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.696308    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"]
Jul 24 16:17:39.696555 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.696528    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62" podUID="6f121208-950f-4c5e-89e4-19e9c5473a0d" containerName="payload-processing" containerID="cri-o://a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f" gracePeriod=30
Jul 24 16:17:39.705866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.705251    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"]
Jul 24 16:17:39.705866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.705443    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz" podUID="f8b74deb-bb21-4665-9722-e39b4d0675b6" containerName="payload-pre-processing" containerID="cri-o://9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96" gracePeriod=30
Jul 24 16:17:39.774283 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.774250    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:39.829937 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.829917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:39.984338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.984318    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:39.987126 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:39.987105    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:40.030848 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.030826    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6tfl\" (UniqueName: \"kubernetes.io/projected/6f121208-950f-4c5e-89e4-19e9c5473a0d-kube-api-access-h6tfl\") pod \"6f121208-950f-4c5e-89e4-19e9c5473a0d\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") "
Jul 24 16:17:40.030969 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.030878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b74deb-bb21-4665-9722-e39b4d0675b6-plugins-config-volume\") pod \"f8b74deb-bb21-4665-9722-e39b4d0675b6\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") "
Jul 24 16:17:40.030969 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.030897    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f121208-950f-4c5e-89e4-19e9c5473a0d-plugins-config-volume\") pod \"6f121208-950f-4c5e-89e4-19e9c5473a0d\" (UID: \"6f121208-950f-4c5e-89e4-19e9c5473a0d\") "
Jul 24 16:17:40.030969 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.030957    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzj5b\" (UniqueName: \"kubernetes.io/projected/f8b74deb-bb21-4665-9722-e39b4d0675b6-kube-api-access-rzj5b\") pod \"f8b74deb-bb21-4665-9722-e39b4d0675b6\" (UID: \"f8b74deb-bb21-4665-9722-e39b4d0675b6\") "
Jul 24 16:17:40.031282 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.031255    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f121208-950f-4c5e-89e4-19e9c5473a0d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f121208-950f-4c5e-89e4-19e9c5473a0d" (UID: "6f121208-950f-4c5e-89e4-19e9c5473a0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:17:40.031370 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.031319    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8b74deb-bb21-4665-9722-e39b4d0675b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8b74deb-bb21-4665-9722-e39b4d0675b6" (UID: "f8b74deb-bb21-4665-9722-e39b4d0675b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:17:40.032905 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.032885    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f121208-950f-4c5e-89e4-19e9c5473a0d-kube-api-access-h6tfl" (OuterVolumeSpecName: "kube-api-access-h6tfl") pod "6f121208-950f-4c5e-89e4-19e9c5473a0d" (UID: "6f121208-950f-4c5e-89e4-19e9c5473a0d"). InnerVolumeSpecName "kube-api-access-h6tfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:17:40.033011 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.032987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8b74deb-bb21-4665-9722-e39b4d0675b6-kube-api-access-rzj5b" (OuterVolumeSpecName: "kube-api-access-rzj5b") pod "f8b74deb-bb21-4665-9722-e39b4d0675b6" (UID: "f8b74deb-bb21-4665-9722-e39b4d0675b6"). InnerVolumeSpecName "kube-api-access-rzj5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:17:40.131489 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.131453    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b74deb-bb21-4665-9722-e39b4d0675b6-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:17:40.131489 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.131487    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f121208-950f-4c5e-89e4-19e9c5473a0d-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:17:40.131648 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.131501    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzj5b\" (UniqueName: \"kubernetes.io/projected/f8b74deb-bb21-4665-9722-e39b4d0675b6-kube-api-access-rzj5b\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:17:40.131648 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.131515    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6tfl\" (UniqueName: \"kubernetes.io/projected/6f121208-950f-4c5e-89e4-19e9c5473a0d-kube-api-access-h6tfl\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:17:40.552499 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.552474    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8b74deb-bb21-4665-9722-e39b4d0675b6" containerID="9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96" exitCode=0
Jul 24 16:17:40.552655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.552524    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"
Jul 24 16:17:40.552655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.552543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz" event={"ID":"f8b74deb-bb21-4665-9722-e39b4d0675b6","Type":"ContainerDied","Data":"9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96"}
Jul 24 16:17:40.552655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.552600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz" event={"ID":"f8b74deb-bb21-4665-9722-e39b4d0675b6","Type":"ContainerDied","Data":"e83d47bd3ee1127dc132e6781a323f7e15a40fe4c146dfe50c618a587b9b117c"}
Jul 24 16:17:40.552781 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.552651    2577 scope.go:117] "RemoveContainer" containerID="9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96"
Jul 24 16:17:40.554348 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.554325    2577 generic.go:356] "Generic (PLEG): container finished" podID="6f121208-950f-4c5e-89e4-19e9c5473a0d" containerID="a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f" exitCode=0
Jul 24 16:17:40.554436 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.554377    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"
Jul 24 16:17:40.554436 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.554412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62" event={"ID":"6f121208-950f-4c5e-89e4-19e9c5473a0d","Type":"ContainerDied","Data":"a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f"}
Jul 24 16:17:40.554553 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.554467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62" event={"ID":"6f121208-950f-4c5e-89e4-19e9c5473a0d","Type":"ContainerDied","Data":"4cbc964eb456dce5eed2a14af745156f90c0c404ab5a007a8d1fd14b40af52f6"}
Jul 24 16:17:40.562672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.562655    2577 scope.go:117] "RemoveContainer" containerID="9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96"
Jul 24 16:17:40.562915 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:17:40.562895    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96\": container with ID starting with 9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96 not found: ID does not exist" containerID="9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96"
Jul 24 16:17:40.562967 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.562923    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96"} err="failed to get container status \"9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96\": rpc error: code = NotFound desc = could not find container \"9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96\": container with ID starting with 9e0623bacbeeb35baebe066f88ad5550966618b98c46e850431e69fbcd9bed96 not found: ID does not exist"
Jul 24 16:17:40.562967 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.562939    2577 scope.go:117] "RemoveContainer" containerID="a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f"
Jul 24 16:17:40.569801 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.569781    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"]
Jul 24 16:17:40.571388 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.571368    2577 scope.go:117] "RemoveContainer" containerID="a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f"
Jul 24 16:17:40.571930 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:17:40.571898    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f\": container with ID starting with a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f not found: ID does not exist" containerID="a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f"
Jul 24 16:17:40.572025 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.571929    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f"} err="failed to get container status \"a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f\": rpc error: code = NotFound desc = could not find container \"a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f\": container with ID starting with a5b94c921db192a69c7092f5b42a7cd7714fa48e85c4139d766d60999c8df52f not found: ID does not exist"
Jul 24 16:17:40.573495 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.573476    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8426fff8-7c4c686f94-ggp62"]
Jul 24 16:17:40.583042 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.583024    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"]
Jul 24 16:17:40.587944 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:40.587925    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8426fff8-774dcf97c6-fb4mz"]
Jul 24 16:17:42.237930 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:42.237890    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f121208-950f-4c5e-89e4-19e9c5473a0d" path="/var/lib/kubelet/pods/6f121208-950f-4c5e-89e4-19e9c5473a0d/volumes"
Jul 24 16:17:42.238413 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:42.238370    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8b74deb-bb21-4665-9722-e39b4d0675b6" path="/var/lib/kubelet/pods/f8b74deb-bb21-4665-9722-e39b4d0675b6/volumes"
Jul 24 16:17:42.693044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:42.693020    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:17:44.152610 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:44.152580    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"]
Jul 24 16:17:44.157061 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:44.157036    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8426fff8-78gqn"]
Jul 24 16:17:44.157307 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:44.157289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:44.237706 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:44.237681    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51a17c26-afbd-4568-a27d-0402e3821fc9" path="/var/lib/kubelet/pods/51a17c26-afbd-4568-a27d-0402e3821fc9/volumes"
Jul 24 16:17:44.307030 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:44.306980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:44.545699 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:44.545668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:45.026136 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:45.026101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:45.114540 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:45.114513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:45.213555 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:45.213530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:46.592320 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:46.592286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:52.320574 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:52.320542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:52.386130 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:52.386107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:52.473513 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:52.473489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:52.570947 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:52.570882    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:52.670831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:52.670811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:53.023097 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:53.023069    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:55.892839 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:55.892805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:55.994602 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:17:55.994573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:18.973374 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:18.973343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:20.929048 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:20.928972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:21.028088 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:21.028062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:25.392461 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:25.392426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:25.496159 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:25.496119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:25.626227 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:25.626198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:41.504452 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:41.504411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:47.440695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:47.440666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:47.548551 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:47.548524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:47.645171 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:47.645141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:58.171230 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:58.171202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:58.939710 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:18:58.939674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:05.869957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:05.869927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:05.980980 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:05.980948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:06.072706 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:06.072677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:06.171896 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:06.171824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:06.961841 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:06.961814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:07.062711 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:07.062684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:07.158729 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:07.158703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:08.704734 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:08.704700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:10.590481 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:10.590448    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:19:14.408074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:14.408045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:14.507800 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:14.507774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:15.081900 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:15.081869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:20.763286 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:20.763253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:20.825354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:20.825327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:20.997983 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:20.997958    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:19:21.009379 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:21.009353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:21.333654 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:21.333588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:21.431344 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:21.431317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:24.556479 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:24.556441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:24.654178 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:24.654150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:24.757337 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:24.757312    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:25.062819 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.062785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:25.238506 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.238479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:25.332189 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332128    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"]
Jul 24 16:19:25.332466 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332451    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f121208-950f-4c5e-89e4-19e9c5473a0d" containerName="payload-processing"
Jul 24 16:19:25.332509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332468    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f121208-950f-4c5e-89e4-19e9c5473a0d" containerName="payload-processing"
Jul 24 16:19:25.332509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332481    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51a17c26-afbd-4568-a27d-0402e3821fc9" containerName="revoke-keys"
Jul 24 16:19:25.332509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332487    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a17c26-afbd-4568-a27d-0402e3821fc9" containerName="revoke-keys"
Jul 24 16:19:25.332509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332496    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8b74deb-bb21-4665-9722-e39b4d0675b6" containerName="payload-pre-processing"
Jul 24 16:19:25.332509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332502    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8b74deb-bb21-4665-9722-e39b4d0675b6" containerName="payload-pre-processing"
Jul 24 16:19:25.332682 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332564    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f121208-950f-4c5e-89e4-19e9c5473a0d" containerName="payload-processing"
Jul 24 16:19:25.332682 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332573    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="51a17c26-afbd-4568-a27d-0402e3821fc9" containerName="revoke-keys"
Jul 24 16:19:25.332682 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.332580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8b74deb-bb21-4665-9722-e39b4d0675b6" containerName="payload-pre-processing"
Jul 24 16:19:25.335767 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.335744    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.337030 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.337002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:25.337681 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.337662    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e1fef210\""
Jul 24 16:19:25.342751 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.342731    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"]
Jul 24 16:19:25.369082 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.369061    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"]
Jul 24 16:19:25.372117 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.372101    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.374092 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.374078    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e1fef210-dockercfg-pv6kt\""
Jul 24 16:19:25.383849 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.383828    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"]
Jul 24 16:19:25.408653 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.408634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn8rp\" (UniqueName: \"kubernetes.io/projected/b4534cfa-f821-4dfb-90ea-da747fc41620-kube-api-access-tn8rp\") pod \"payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.408741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.408717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4534cfa-f821-4dfb-90ea-da747fc41620-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.509730 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.509698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwx5j\" (UniqueName: \"kubernetes.io/projected/2e8fa801-46a2-477f-93d0-88e41d5121ee-kube-api-access-bwx5j\") pod \"payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") " pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.509866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.509748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn8rp\" (UniqueName: \"kubernetes.io/projected/b4534cfa-f821-4dfb-90ea-da747fc41620-kube-api-access-tn8rp\") pod \"payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.509866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.509853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4534cfa-f821-4dfb-90ea-da747fc41620-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.509954 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.509882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8fa801-46a2-477f-93d0-88e41d5121ee-plugins-config-volume\") pod \"payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") " pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.510539 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.510522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4534cfa-f821-4dfb-90ea-da747fc41620-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.517573 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.517539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn8rp\" (UniqueName: \"kubernetes.io/projected/b4534cfa-f821-4dfb-90ea-da747fc41620-kube-api-access-tn8rp\") pod \"payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.611304 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.611235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8fa801-46a2-477f-93d0-88e41d5121ee-plugins-config-volume\") pod \"payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") " pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.611304 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.611271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwx5j\" (UniqueName: \"kubernetes.io/projected/2e8fa801-46a2-477f-93d0-88e41d5121ee-kube-api-access-bwx5j\") pod \"payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") " pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.611948 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.611931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8fa801-46a2-477f-93d0-88e41d5121ee-plugins-config-volume\") pod \"payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") " pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.618398 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.618377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwx5j\" (UniqueName: \"kubernetes.io/projected/2e8fa801-46a2-477f-93d0-88e41d5121ee-kube-api-access-bwx5j\") pod \"payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") " pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.647033 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.647014    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:25.681113 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.681084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:25.828748 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.828727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"]
Jul 24 16:19:25.830188 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:19:25.830159    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8fa801_46a2_477f_93d0_88e41d5121ee.slice/crio-0a0f07241fc31832bb1e6ccb6ba7f826df294b15add728c638b825f6496d19ef WatchSource:0}: Error finding container 0a0f07241fc31832bb1e6ccb6ba7f826df294b15add728c638b825f6496d19ef: Status 404 returned error can't find the container with id 0a0f07241fc31832bb1e6ccb6ba7f826df294b15add728c638b825f6496d19ef
Jul 24 16:19:25.992417 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:25.992393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"]
Jul 24 16:19:25.994321 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:19:25.994293    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4534cfa_f821_4dfb_90ea_da747fc41620.slice/crio-feda0ca6cca4636fe8fb009b6f78f6e61571fde87f542030c469c399d593bd1c WatchSource:0}: Error finding container feda0ca6cca4636fe8fb009b6f78f6e61571fde87f542030c469c399d593bd1c: Status 404 returned error can't find the container with id feda0ca6cca4636fe8fb009b6f78f6e61571fde87f542030c469c399d593bd1c
Jul 24 16:19:26.107291 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.107267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd" event={"ID":"b4534cfa-f821-4dfb-90ea-da747fc41620","Type":"ContainerStarted","Data":"d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925"}
Jul 24 16:19:26.107424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.107313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd" event={"ID":"b4534cfa-f821-4dfb-90ea-da747fc41620","Type":"ContainerStarted","Data":"feda0ca6cca4636fe8fb009b6f78f6e61571fde87f542030c469c399d593bd1c"}
Jul 24 16:19:26.107489 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.107421    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:26.108802 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.108778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz" event={"ID":"2e8fa801-46a2-477f-93d0-88e41d5121ee","Type":"ContainerStarted","Data":"a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28"}
Jul 24 16:19:26.108903 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.108824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:26.108903 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.108841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz" event={"ID":"2e8fa801-46a2-477f-93d0-88e41d5121ee","Type":"ContainerStarted","Data":"0a0f07241fc31832bb1e6ccb6ba7f826df294b15add728c638b825f6496d19ef"}
Jul 24 16:19:26.126328 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.126260    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd" podStartSLOduration=1.126247393 podStartE2EDuration="1.126247393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:19:26.124761038 +0000 UTC m=+1948.452325576" watchObservedRunningTime="2026-07-24 16:19:26.126247393 +0000 UTC m=+1948.453811934"
Jul 24 16:19:26.153190 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:26.153153    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz" podStartSLOduration=1.153141406 podStartE2EDuration="1.153141406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:19:26.149802726 +0000 UTC m=+1948.477367283" watchObservedRunningTime="2026-07-24 16:19:26.153141406 +0000 UTC m=+1948.480705944"
Jul 24 16:19:29.004245 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:29.004207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:29.628680 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:29.628650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:29.730861 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:29.730831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:30.060419 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:30.060391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:30.721164 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:30.721132    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:19:35.842927 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:35.842891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:35.910070 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:35.910045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:36.184807 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.184739    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"]
Jul 24 16:19:36.187978 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.187958    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:36.190010 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.189980    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:19:36.190105 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.190088    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:19:36.195080 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.195059    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"]
Jul 24 16:19:36.291118 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.291098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e247a7e6-2a6c-4616-a64a-c839bd9fbd37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg\" (UID: \"e247a7e6-2a6c-4616-a64a-c839bd9fbd37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:36.392236 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.392214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e247a7e6-2a6c-4616-a64a-c839bd9fbd37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg\" (UID: \"e247a7e6-2a6c-4616-a64a-c839bd9fbd37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:36.392782 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.392767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e247a7e6-2a6c-4616-a64a-c839bd9fbd37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg\" (UID: \"e247a7e6-2a6c-4616-a64a-c839bd9fbd37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:36.499107 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.499047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:36.624628 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:36.624590    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"]
Jul 24 16:19:36.625831 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:19:36.625809    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode247a7e6_2a6c_4616_a64a_c839bd9fbd37.slice/crio-dd3884b11625bd33375420270b89043e3b452ae63042b986d789bcda51fb99b1 WatchSource:0}: Error finding container dd3884b11625bd33375420270b89043e3b452ae63042b986d789bcda51fb99b1: Status 404 returned error can't find the container with id dd3884b11625bd33375420270b89043e3b452ae63042b986d789bcda51fb99b1
Jul 24 16:19:37.116842 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:37.116790    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:37.117140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:37.116858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:37.171121 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:37.171087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg" event={"ID":"e247a7e6-2a6c-4616-a64a-c839bd9fbd37","Type":"ContainerStarted","Data":"a5eea94b5822a8c9eed4988bd54fd7e4f41f32cccf7a239919f8ed2fbb19e5a0"}
Jul 24 16:19:37.171276 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:37.171146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg" event={"ID":"e247a7e6-2a6c-4616-a64a-c839bd9fbd37","Type":"ContainerStarted","Data":"dd3884b11625bd33375420270b89043e3b452ae63042b986d789bcda51fb99b1"}
Jul 24 16:19:38.178284 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:38.178249    2577 generic.go:356] "Generic (PLEG): container finished" podID="e247a7e6-2a6c-4616-a64a-c839bd9fbd37" containerID="a5eea94b5822a8c9eed4988bd54fd7e4f41f32cccf7a239919f8ed2fbb19e5a0" exitCode=0
Jul 24 16:19:38.178666 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:38.178336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg" event={"ID":"e247a7e6-2a6c-4616-a64a-c839bd9fbd37","Type":"ContainerDied","Data":"a5eea94b5822a8c9eed4988bd54fd7e4f41f32cccf7a239919f8ed2fbb19e5a0"}
Jul 24 16:19:39.313162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:39.313140    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:39.413895 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:39.413872    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e247a7e6-2a6c-4616-a64a-c839bd9fbd37-maas-api-service-ca\") pod \"e247a7e6-2a6c-4616-a64a-c839bd9fbd37\" (UID: \"e247a7e6-2a6c-4616-a64a-c839bd9fbd37\") "
Jul 24 16:19:39.414166 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:39.414147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e247a7e6-2a6c-4616-a64a-c839bd9fbd37-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e247a7e6-2a6c-4616-a64a-c839bd9fbd37" (UID: "e247a7e6-2a6c-4616-a64a-c839bd9fbd37"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:39.514298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:39.514243    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e247a7e6-2a6c-4616-a64a-c839bd9fbd37-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.189802 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:40.189773    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"
Jul 24 16:19:40.189802 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:40.189782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg" event={"ID":"e247a7e6-2a6c-4616-a64a-c839bd9fbd37","Type":"ContainerDied","Data":"dd3884b11625bd33375420270b89043e3b452ae63042b986d789bcda51fb99b1"}
Jul 24 16:19:40.190012 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:40.189822    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd3884b11625bd33375420270b89043e3b452ae63042b986d789bcda51fb99b1"
Jul 24 16:19:46.735112 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.735074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:46.755117 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.755087    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"]
Jul 24 16:19:46.755376 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.755322    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz" podUID="2e8fa801-46a2-477f-93d0-88e41d5121ee" containerName="payload-processing" containerID="cri-o://a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28" gracePeriod=30
Jul 24 16:19:46.763247 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.763213    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"]
Jul 24 16:19:46.763492 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.763445    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd" podUID="b4534cfa-f821-4dfb-90ea-da747fc41620" containerName="payload-pre-processing" containerID="cri-o://d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925" gracePeriod=30
Jul 24 16:19:46.834308 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.834284    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:46.936057 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:46.936022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:47.000911 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.000889    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:47.032145 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.032127    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:47.172392 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.172363    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8fa801-46a2-477f-93d0-88e41d5121ee-plugins-config-volume\") pod \"2e8fa801-46a2-477f-93d0-88e41d5121ee\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") "
Jul 24 16:19:47.172535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.172398    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwx5j\" (UniqueName: \"kubernetes.io/projected/2e8fa801-46a2-477f-93d0-88e41d5121ee-kube-api-access-bwx5j\") pod \"2e8fa801-46a2-477f-93d0-88e41d5121ee\" (UID: \"2e8fa801-46a2-477f-93d0-88e41d5121ee\") "
Jul 24 16:19:47.172535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.172462    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4534cfa-f821-4dfb-90ea-da747fc41620-plugins-config-volume\") pod \"b4534cfa-f821-4dfb-90ea-da747fc41620\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") "
Jul 24 16:19:47.172683 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.172533    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tn8rp\" (UniqueName: \"kubernetes.io/projected/b4534cfa-f821-4dfb-90ea-da747fc41620-kube-api-access-tn8rp\") pod \"b4534cfa-f821-4dfb-90ea-da747fc41620\" (UID: \"b4534cfa-f821-4dfb-90ea-da747fc41620\") "
Jul 24 16:19:47.172868 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.172829    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e8fa801-46a2-477f-93d0-88e41d5121ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e8fa801-46a2-477f-93d0-88e41d5121ee" (UID: "2e8fa801-46a2-477f-93d0-88e41d5121ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:47.172940 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.172878    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4534cfa-f821-4dfb-90ea-da747fc41620-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4534cfa-f821-4dfb-90ea-da747fc41620" (UID: "b4534cfa-f821-4dfb-90ea-da747fc41620"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:47.174553 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.174531    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e8fa801-46a2-477f-93d0-88e41d5121ee-kube-api-access-bwx5j" (OuterVolumeSpecName: "kube-api-access-bwx5j") pod "2e8fa801-46a2-477f-93d0-88e41d5121ee" (UID: "2e8fa801-46a2-477f-93d0-88e41d5121ee"). InnerVolumeSpecName "kube-api-access-bwx5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:47.174625 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.174564    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4534cfa-f821-4dfb-90ea-da747fc41620-kube-api-access-tn8rp" (OuterVolumeSpecName: "kube-api-access-tn8rp") pod "b4534cfa-f821-4dfb-90ea-da747fc41620" (UID: "b4534cfa-f821-4dfb-90ea-da747fc41620"). InnerVolumeSpecName "kube-api-access-tn8rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:47.229830 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.229806    2577 generic.go:356] "Generic (PLEG): container finished" podID="b4534cfa-f821-4dfb-90ea-da747fc41620" containerID="d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925" exitCode=0
Jul 24 16:19:47.229931 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.229845    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"
Jul 24 16:19:47.229931 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.229895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd" event={"ID":"b4534cfa-f821-4dfb-90ea-da747fc41620","Type":"ContainerDied","Data":"d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925"}
Jul 24 16:19:47.230051 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.229953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd" event={"ID":"b4534cfa-f821-4dfb-90ea-da747fc41620","Type":"ContainerDied","Data":"feda0ca6cca4636fe8fb009b6f78f6e61571fde87f542030c469c399d593bd1c"}
Jul 24 16:19:47.230051 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.229985    2577 scope.go:117] "RemoveContainer" containerID="d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925"
Jul 24 16:19:47.231261 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.231237    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e8fa801-46a2-477f-93d0-88e41d5121ee" containerID="a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28" exitCode=0
Jul 24 16:19:47.231343 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.231276    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"
Jul 24 16:19:47.231343 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.231298    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz" event={"ID":"2e8fa801-46a2-477f-93d0-88e41d5121ee","Type":"ContainerDied","Data":"a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28"}
Jul 24 16:19:47.231343 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.231336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz" event={"ID":"2e8fa801-46a2-477f-93d0-88e41d5121ee","Type":"ContainerDied","Data":"0a0f07241fc31832bb1e6ccb6ba7f826df294b15add728c638b825f6496d19ef"}
Jul 24 16:19:47.239972 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.239956    2577 scope.go:117] "RemoveContainer" containerID="d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925"
Jul 24 16:19:47.240230 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:19:47.240209    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925\": container with ID starting with d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925 not found: ID does not exist" containerID="d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925"
Jul 24 16:19:47.240325 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.240235    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925"} err="failed to get container status \"d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925\": rpc error: code = NotFound desc = could not find container \"d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925\": container with ID starting with d56dbb3a616ed3b9ca1f3367e758e7efaa14700e3745c533502a73c01afc5925 not found: ID does not exist"
Jul 24 16:19:47.240325 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.240251    2577 scope.go:117] "RemoveContainer" containerID="a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28"
Jul 24 16:19:47.248176 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.248162    2577 scope.go:117] "RemoveContainer" containerID="a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28"
Jul 24 16:19:47.248479 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:19:47.248450    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28\": container with ID starting with a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28 not found: ID does not exist" containerID="a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28"
Jul 24 16:19:47.248538 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.248486    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28"} err="failed to get container status \"a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28\": rpc error: code = NotFound desc = could not find container \"a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28\": container with ID starting with a7246d93397f788936e3801ddb870747ea52c077d3ff9b03c54e34fa2ea64b28 not found: ID does not exist"
Jul 24 16:19:47.251810 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.251792    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"]
Jul 24 16:19:47.253775 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.253755    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e1fef210-5db448fc96-wxgcd"]
Jul 24 16:19:47.263196 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.263176    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"]
Jul 24 16:19:47.267145 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.267128    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e1fef210-bb74c7974-zctrz"]
Jul 24 16:19:47.273649 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.273634    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8fa801-46a2-477f-93d0-88e41d5121ee-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:19:47.273703 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.273652    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwx5j\" (UniqueName: \"kubernetes.io/projected/2e8fa801-46a2-477f-93d0-88e41d5121ee-kube-api-access-bwx5j\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:19:47.273703 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.273661    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4534cfa-f821-4dfb-90ea-da747fc41620-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:19:47.273703 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:47.273670    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn8rp\" (UniqueName: \"kubernetes.io/projected/b4534cfa-f821-4dfb-90ea-da747fc41620-kube-api-access-tn8rp\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:19:48.239245 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:48.239156    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e8fa801-46a2-477f-93d0-88e41d5121ee" path="/var/lib/kubelet/pods/2e8fa801-46a2-477f-93d0-88e41d5121ee/volumes"
Jul 24 16:19:48.239685 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:48.239607    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4534cfa-f821-4dfb-90ea-da747fc41620" path="/var/lib/kubelet/pods/b4534cfa-f821-4dfb-90ea-da747fc41620/volumes"
Jul 24 16:19:49.794241 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:49.794209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:19:51.254518 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:51.254481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:51.260489 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:51.260453    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"]
Jul 24 16:19:51.262676 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:51.262656    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1fef210-9t6qg"]
Jul 24 16:19:51.652685 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:51.652653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:52.238458 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:52.238422    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e247a7e6-2a6c-4616-a64a-c839bd9fbd37" path="/var/lib/kubelet/pods/e247a7e6-2a6c-4616-a64a-c839bd9fbd37/volumes"
Jul 24 16:19:57.521126 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:57.521092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:57.586630 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:57.586587    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:57.770039 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:57.770000    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:57.988998 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:57.988928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:58.133029 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:58.132995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:58.219382 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:58.219352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:58.341093 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:19:58.341068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:04.099320 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:04.099291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:04.204682 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:04.204655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:04.299528 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:04.299497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:04.463177 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:04.463111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:10.115773 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:10.115738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:10.218711 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:10.218675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:16.837570 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:16.837529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:17.266040 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:17.265959    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:17.452419 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:17.452383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:19.808385 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:19.808352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:19.887750 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:19.887720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:19.959302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:19.959266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:20.506195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.506168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:20.599162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.599132    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:20.666350 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666323    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"]
Jul 24 16:20:20.666652 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666640    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4534cfa-f821-4dfb-90ea-da747fc41620" containerName="payload-pre-processing"
Jul 24 16:20:20.666704 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666654    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4534cfa-f821-4dfb-90ea-da747fc41620" containerName="payload-pre-processing"
Jul 24 16:20:20.666704 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666667    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e8fa801-46a2-477f-93d0-88e41d5121ee" containerName="payload-processing"
Jul 24 16:20:20.666704 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666673    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e8fa801-46a2-477f-93d0-88e41d5121ee" containerName="payload-processing"
Jul 24 16:20:20.666704 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666694    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e247a7e6-2a6c-4616-a64a-c839bd9fbd37" containerName="revoke-keys"
Jul 24 16:20:20.666704 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666700    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e247a7e6-2a6c-4616-a64a-c839bd9fbd37" containerName="revoke-keys"
Jul 24 16:20:20.666882 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666751    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4534cfa-f821-4dfb-90ea-da747fc41620" containerName="payload-pre-processing"
Jul 24 16:20:20.666882 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666760    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e247a7e6-2a6c-4616-a64a-c839bd9fbd37" containerName="revoke-keys"
Jul 24 16:20:20.666882 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.666767    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e8fa801-46a2-477f-93d0-88e41d5121ee" containerName="payload-processing"
Jul 24 16:20:20.669762 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.669743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.672741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.672722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b13121e3\""
Jul 24 16:20:20.672741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.672734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b13121e3-dockercfg-p79jd\""
Jul 24 16:20:20.688642 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.688604    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"]
Jul 24 16:20:20.718714 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.718690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c2hx\" (UniqueName: \"kubernetes.io/projected/e774994b-950b-41da-839e-61835bd1bc34-kube-api-access-7c2hx\") pod \"payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") " pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.718831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.718730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e774994b-950b-41da-839e-61835bd1bc34-plugins-config-volume\") pod \"payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") " pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.727124 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.727100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:20.820008 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.819985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c2hx\" (UniqueName: \"kubernetes.io/projected/e774994b-950b-41da-839e-61835bd1bc34-kube-api-access-7c2hx\") pod \"payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") " pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.820334 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.820015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e774994b-950b-41da-839e-61835bd1bc34-plugins-config-volume\") pod \"payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") " pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.820673 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.820654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e774994b-950b-41da-839e-61835bd1bc34-plugins-config-volume\") pod \"payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") " pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.828380 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.828358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c2hx\" (UniqueName: \"kubernetes.io/projected/e774994b-950b-41da-839e-61835bd1bc34-kube-api-access-7c2hx\") pod \"payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") " pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:20.980280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:20.980258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:21.329335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:21.329310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"]
Jul 24 16:20:21.330123 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:20:21.330102    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode774994b_950b_41da_839e_61835bd1bc34.slice/crio-92721e7d40ff4beccf1b24465440c7e856999ce5aa804b042ff644ec8da42293 WatchSource:0}: Error finding container 92721e7d40ff4beccf1b24465440c7e856999ce5aa804b042ff644ec8da42293: Status 404 returned error can't find the container with id 92721e7d40ff4beccf1b24465440c7e856999ce5aa804b042ff644ec8da42293
Jul 24 16:20:21.421547 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:21.421519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb" event={"ID":"e774994b-950b-41da-839e-61835bd1bc34","Type":"ContainerStarted","Data":"968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d"}
Jul 24 16:20:21.421675 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:21.421578    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb" event={"ID":"e774994b-950b-41da-839e-61835bd1bc34","Type":"ContainerStarted","Data":"92721e7d40ff4beccf1b24465440c7e856999ce5aa804b042ff644ec8da42293"}
Jul 24 16:20:21.421725 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:21.421713    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:21.455046 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:21.455000    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb" podStartSLOduration=1.454984852 podStartE2EDuration="1.454984852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:21.453829323 +0000 UTC m=+2003.781393881" watchObservedRunningTime="2026-07-24 16:20:21.454984852 +0000 UTC m=+2003.782549388"
Jul 24 16:20:24.275144 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:24.275115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:25.196267 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:25.196234    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:20:32.428772 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:32.428740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:20:37.918147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.918107    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"]
Jul 24 16:20:37.921672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.921655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:37.923703 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.923679    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b13121e3\"/\"kube-root-ca.crt\""
Jul 24 16:20:37.924314 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.924291    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b13121e3\"/\"default-dockercfg-g7cb4\""
Jul 24 16:20:37.924314 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.924302    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b13121e3\"/\"openshift-service-ca.crt\""
Jul 24 16:20:37.924472 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.924345    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b13121e3\"/\"e2e-lifecycle-model-b13121e3-kserve-self-signed-certs\""
Jul 24 16:20:37.932069 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:37.932046    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"]
Jul 24 16:20:38.049019 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.048994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-home\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.049113 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.049023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-dshm\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.049113 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.049057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ksln\" (UniqueName: \"kubernetes.io/projected/0128ddae-d343-4267-b983-60f014522c71-kube-api-access-7ksln\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.049113 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.049104    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-tmp-dir\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.049221 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.049163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-model-cache\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.049221 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.049197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0128ddae-d343-4267-b983-60f014522c71-tls-certs\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.150548 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.150521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-model-cache\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.150766 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.150741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0128ddae-d343-4267-b983-60f014522c71-tls-certs\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.150885 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.150811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-home\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.150885 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.150844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-dshm\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.151009 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.150903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ksln\" (UniqueName: \"kubernetes.io/projected/0128ddae-d343-4267-b983-60f014522c71-kube-api-access-7ksln\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.151009 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.150977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-model-cache\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.151123 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.151087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-home\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.151227 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.151202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-tmp-dir\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.151314 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.151293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-tmp-dir\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.152989 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.152971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-dshm\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.153411 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.153396    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0128ddae-d343-4267-b983-60f014522c71-tls-certs\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.157845 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.157821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ksln\" (UniqueName: \"kubernetes.io/projected/0128ddae-d343-4267-b983-60f014522c71-kube-api-access-7ksln\") pod \"e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") " pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.188302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.188249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:38.232360 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.232343    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:38.363069 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.363048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"]
Jul 24 16:20:38.363888 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:20:38.363866    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0128ddae_d343_4267_b983_60f014522c71.slice/crio-f4f7edf371ccf7908f0f24a23a14b1c1b0ea2bf9df8dd7d6f863a6a820087a0d WatchSource:0}: Error finding container f4f7edf371ccf7908f0f24a23a14b1c1b0ea2bf9df8dd7d6f863a6a820087a0d: Status 404 returned error can't find the container with id f4f7edf371ccf7908f0f24a23a14b1c1b0ea2bf9df8dd7d6f863a6a820087a0d
Jul 24 16:20:38.511893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.511822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l" event={"ID":"0128ddae-d343-4267-b983-60f014522c71","Type":"ContainerStarted","Data":"5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625"}
Jul 24 16:20:38.511893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.511863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l" event={"ID":"0128ddae-d343-4267-b983-60f014522c71","Type":"ContainerStarted","Data":"f4f7edf371ccf7908f0f24a23a14b1c1b0ea2bf9df8dd7d6f863a6a820087a0d"}
Jul 24 16:20:38.528685 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:38.528644    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l" podStartSLOduration=1.528630486 podStartE2EDuration="1.528630486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:38.526586683 +0000 UTC m=+2020.854151237" watchObservedRunningTime="2026-07-24 16:20:38.528630486 +0000 UTC m=+2020.856195020"
Jul 24 16:20:41.307383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:41.307338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:41.791646 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:41.791609    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:20:48.239148 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:48.239114    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:48.239677 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:48.239161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:48.245666 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:48.245644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:48.576552 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:48.576520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:20:54.495505 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:54.495471    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:20:55.126421 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:20:55.126389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:12.401496 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:12.401465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:21:14.153018 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.152988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:14.181543 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.181509    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"]
Jul 24 16:21:14.181881 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.181828    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb" podUID="e774994b-950b-41da-839e-61835bd1bc34" containerName="payload-processing" containerID="cri-o://968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d" gracePeriod=30
Jul 24 16:21:14.437041 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.437016    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:21:14.527904 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.527876    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e774994b-950b-41da-839e-61835bd1bc34-plugins-config-volume\") pod \"e774994b-950b-41da-839e-61835bd1bc34\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") "
Jul 24 16:21:14.528044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.527926    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c2hx\" (UniqueName: \"kubernetes.io/projected/e774994b-950b-41da-839e-61835bd1bc34-kube-api-access-7c2hx\") pod \"e774994b-950b-41da-839e-61835bd1bc34\" (UID: \"e774994b-950b-41da-839e-61835bd1bc34\") "
Jul 24 16:21:14.528233 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.528212    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e774994b-950b-41da-839e-61835bd1bc34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e774994b-950b-41da-839e-61835bd1bc34" (UID: "e774994b-950b-41da-839e-61835bd1bc34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:21:14.529957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.529938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e774994b-950b-41da-839e-61835bd1bc34-kube-api-access-7c2hx" (OuterVolumeSpecName: "kube-api-access-7c2hx") pod "e774994b-950b-41da-839e-61835bd1bc34" (UID: "e774994b-950b-41da-839e-61835bd1bc34"). InnerVolumeSpecName "kube-api-access-7c2hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:14.628608 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.628581    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e774994b-950b-41da-839e-61835bd1bc34-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:14.628751 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.628603    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c2hx\" (UniqueName: \"kubernetes.io/projected/e774994b-950b-41da-839e-61835bd1bc34-kube-api-access-7c2hx\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:14.720346 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.720272    2577 generic.go:356] "Generic (PLEG): container finished" podID="e774994b-950b-41da-839e-61835bd1bc34" containerID="968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d" exitCode=0
Jul 24 16:21:14.720346 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.720309    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"
Jul 24 16:21:14.720545 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.720357    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb" event={"ID":"e774994b-950b-41da-839e-61835bd1bc34","Type":"ContainerDied","Data":"968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d"}
Jul 24 16:21:14.720545 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.720411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb" event={"ID":"e774994b-950b-41da-839e-61835bd1bc34","Type":"ContainerDied","Data":"92721e7d40ff4beccf1b24465440c7e856999ce5aa804b042ff644ec8da42293"}
Jul 24 16:21:14.720545 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.720442    2577 scope.go:117] "RemoveContainer" containerID="968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d"
Jul 24 16:21:14.729580 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.729558    2577 scope.go:117] "RemoveContainer" containerID="968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d"
Jul 24 16:21:14.729838 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:21:14.729817    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d\": container with ID starting with 968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d not found: ID does not exist" containerID="968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d"
Jul 24 16:21:14.729896 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.729847    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d"} err="failed to get container status \"968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d\": rpc error: code = NotFound desc = could not find container \"968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d\": container with ID starting with 968339c5437c4c29f11a89a197e1f7db4a0f174502a62c6d91bc3db97e7dc74d not found: ID does not exist"
Jul 24 16:21:14.741025 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.741004    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"]
Jul 24 16:21:14.748000 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:14.747981    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b13121e3-7b66576fdb-4mwjb"]
Jul 24 16:21:15.273828 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:15.273800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:16.238471 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:16.238438    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e774994b-950b-41da-839e-61835bd1bc34" path="/var/lib/kubelet/pods/e774994b-950b-41da-839e-61835bd1bc34/volumes"
Jul 24 16:21:18.677922 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:18.677885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:21.795359 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:21.795282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:22.899391 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:22.899356    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:21:26.774513 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:26.774474    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"]
Jul 24 16:21:26.867754 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:26.867719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:41.794360 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:41.794310    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l" podUID="0128ddae-d343-4267-b983-60f014522c71" containerName="main" containerID="cri-o://5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625" gracePeriod=45
Jul 24 16:21:42.033544 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.033519    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:21:42.132302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132222    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-home\") pod \"0128ddae-d343-4267-b983-60f014522c71\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") "
Jul 24 16:21:42.132302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132274    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0128ddae-d343-4267-b983-60f014522c71-tls-certs\") pod \"0128ddae-d343-4267-b983-60f014522c71\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") "
Jul 24 16:21:42.132520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132309    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-dshm\") pod \"0128ddae-d343-4267-b983-60f014522c71\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") "
Jul 24 16:21:42.132520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132332    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ksln\" (UniqueName: \"kubernetes.io/projected/0128ddae-d343-4267-b983-60f014522c71-kube-api-access-7ksln\") pod \"0128ddae-d343-4267-b983-60f014522c71\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") "
Jul 24 16:21:42.132520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-tmp-dir\") pod \"0128ddae-d343-4267-b983-60f014522c71\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") "
Jul 24 16:21:42.132520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-model-cache\") pod \"0128ddae-d343-4267-b983-60f014522c71\" (UID: \"0128ddae-d343-4267-b983-60f014522c71\") "
Jul 24 16:21:42.132738 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132516    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-home" (OuterVolumeSpecName: "home") pod "0128ddae-d343-4267-b983-60f014522c71" (UID: "0128ddae-d343-4267-b983-60f014522c71"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:42.132738 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132674    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0128ddae-d343-4267-b983-60f014522c71" (UID: "0128ddae-d343-4267-b983-60f014522c71"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:42.132843 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132751    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-model-cache" (OuterVolumeSpecName: "model-cache") pod "0128ddae-d343-4267-b983-60f014522c71" (UID: "0128ddae-d343-4267-b983-60f014522c71"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:42.132843 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.132765    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:42.134572 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.134551    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-dshm" (OuterVolumeSpecName: "dshm") pod "0128ddae-d343-4267-b983-60f014522c71" (UID: "0128ddae-d343-4267-b983-60f014522c71"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:42.134572 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.134566    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0128ddae-d343-4267-b983-60f014522c71-kube-api-access-7ksln" (OuterVolumeSpecName: "kube-api-access-7ksln") pod "0128ddae-d343-4267-b983-60f014522c71" (UID: "0128ddae-d343-4267-b983-60f014522c71"). InnerVolumeSpecName "kube-api-access-7ksln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:42.134691 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.134630    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0128ddae-d343-4267-b983-60f014522c71-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0128ddae-d343-4267-b983-60f014522c71" (UID: "0128ddae-d343-4267-b983-60f014522c71"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:21:42.234035 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.234009    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0128ddae-d343-4267-b983-60f014522c71-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:42.234035 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.234031    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:42.234181 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.234045    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ksln\" (UniqueName: \"kubernetes.io/projected/0128ddae-d343-4267-b983-60f014522c71-kube-api-access-7ksln\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:42.234181 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.234058    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:42.234181 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.234072    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0128ddae-d343-4267-b983-60f014522c71-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:21:42.873175 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.873143    2577 generic.go:356] "Generic (PLEG): container finished" podID="0128ddae-d343-4267-b983-60f014522c71" containerID="5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625" exitCode=0
Jul 24 16:21:42.873585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.873184    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"
Jul 24 16:21:42.873585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.873230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l" event={"ID":"0128ddae-d343-4267-b983-60f014522c71","Type":"ContainerDied","Data":"5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625"}
Jul 24 16:21:42.873585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.873273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l" event={"ID":"0128ddae-d343-4267-b983-60f014522c71","Type":"ContainerDied","Data":"f4f7edf371ccf7908f0f24a23a14b1c1b0ea2bf9df8dd7d6f863a6a820087a0d"}
Jul 24 16:21:42.873585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.873294    2577 scope.go:117] "RemoveContainer" containerID="5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625"
Jul 24 16:21:42.881738 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.881722    2577 scope.go:117] "RemoveContainer" containerID="5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625"
Jul 24 16:21:42.882001 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:21:42.881979    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625\": container with ID starting with 5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625 not found: ID does not exist" containerID="5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625"
Jul 24 16:21:42.882061 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.882006    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625"} err="failed to get container status \"5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625\": rpc error: code = NotFound desc = could not find container \"5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625\": container with ID starting with 5cf4e61a790a42b2e275e506817bce4cda095a672cad69ac9f57c7dffd9fe625 not found: ID does not exist"
Jul 24 16:21:42.891430 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.891403    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"]
Jul 24 16:21:42.896853 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:42.896833    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b13121e3/e2e-lifecycle-model-b13121e3-kserve-6677d89947-8md6l"]
Jul 24 16:21:44.238030 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:44.238000    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0128ddae-d343-4267-b983-60f014522c71" path="/var/lib/kubelet/pods/0128ddae-d343-4267-b983-60f014522c71/volumes"
Jul 24 16:21:58.405475 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:58.405446    2577 scope.go:117] "RemoveContainer" containerID="bde5013d0f45fca848fe580e2ec4623fdd42887d2177cd81577bc656eed19b5a"
Jul 24 16:21:58.413502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:21:58.413485    2577 scope.go:117] "RemoveContainer" containerID="7635e87568b8e2a78fe88a38e7f090a7619a12a0cd401a0c38852effde0c05bf"
Jul 24 16:22:03.329105 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:03.329065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:04.244228 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:04.244196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:09.386529 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:09.386494    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:09.488109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:09.488080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:10.018499 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:10.018467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:10.119376 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:10.119353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:10.294074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:10.294044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:10.524184 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:10.524158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:11.562034 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:11.562000    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:11.662516 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:11.662483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:11.762782 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:11.762756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:12.451653 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:12.451603    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:12.536652 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:12.536599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:12.636854 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:12.636824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:13.975448 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:13.975418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:14.399763 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:14.399731    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:22:19.805118 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:19.805086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:19.867740 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:19.867712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:20.481735 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:20.481703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:24.697333 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:24.697304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:22:26.781345 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:26.781293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:26.878662 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:26.878608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:26.975340 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:26.975314    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:37.463579 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:37.463543    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:37.804309 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:37.804286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:58.442262 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:58.442235    2577 scope.go:117] "RemoveContainer" containerID="f382133e558dc33299b6b7a453f070c4c1b13a00e3c36828d4f2d81eedf5a3ae"
Jul 24 16:22:58.450281 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:58.450265    2577 scope.go:117] "RemoveContainer" containerID="5a5ba9490a46c2a038cae30cd39998fde5f8d01e137553e1aedbd616802fdc24"
Jul 24 16:22:58.737081 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:22:58.737016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:04.969059 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:04.969021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:05.073999 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:05.073967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:05.307136 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:05.307105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:05.628045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:05.627953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:05.935768 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:05.935675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:08.681870 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:08.681815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:08.764525 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:08.764495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:08.834226 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:08.834199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:09.003166 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.003065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:09.368927 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.368886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:09.457832 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.457799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:09.505338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505311    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"]
Jul 24 16:23:09.505759 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505744    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e774994b-950b-41da-839e-61835bd1bc34" containerName="payload-processing"
Jul 24 16:23:09.505810 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505761    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e774994b-950b-41da-839e-61835bd1bc34" containerName="payload-processing"
Jul 24 16:23:09.505810 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505781    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0128ddae-d343-4267-b983-60f014522c71" containerName="main"
Jul 24 16:23:09.505810 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505787    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0128ddae-d343-4267-b983-60f014522c71" containerName="main"
Jul 24 16:23:09.505905 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505839    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0128ddae-d343-4267-b983-60f014522c71" containerName="main"
Jul 24 16:23:09.505905 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.505850    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e774994b-950b-41da-839e-61835bd1bc34" containerName="payload-processing"
Jul 24 16:23:09.508713 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.508698    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.511551 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.511530    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e12d61-dockercfg-pg6j2\""
Jul 24 16:23:09.511686 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.511672    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e12d61\""
Jul 24 16:23:09.523584 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.523564    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"]
Jul 24 16:23:09.669952 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.669877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wcrs\" (UniqueName: \"kubernetes.io/projected/1a38668f-e98c-4188-88d1-08fab5627a90-kube-api-access-6wcrs\") pod \"payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.670092 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.669984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a38668f-e98c-4188-88d1-08fab5627a90-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.771139 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.771111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a38668f-e98c-4188-88d1-08fab5627a90-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.771469 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.771161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wcrs\" (UniqueName: \"kubernetes.io/projected/1a38668f-e98c-4188-88d1-08fab5627a90-kube-api-access-6wcrs\") pod \"payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.771827 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.771807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a38668f-e98c-4188-88d1-08fab5627a90-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.779105 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.779088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wcrs\" (UniqueName: \"kubernetes.io/projected/1a38668f-e98c-4188-88d1-08fab5627a90-kube-api-access-6wcrs\") pod \"payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.818045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.818023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:09.944503 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:09.944476    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"]
Jul 24 16:23:09.945984 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:23:09.945954    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a38668f_e98c_4188_88d1_08fab5627a90.slice/crio-a8a76a5ccc7da83317e9892c2f319fcf85bd2c4f18f521f0f620dde42c63e1ce WatchSource:0}: Error finding container a8a76a5ccc7da83317e9892c2f319fcf85bd2c4f18f521f0f620dde42c63e1ce: Status 404 returned error can't find the container with id a8a76a5ccc7da83317e9892c2f319fcf85bd2c4f18f521f0f620dde42c63e1ce
Jul 24 16:23:10.346848 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:10.346819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8" event={"ID":"1a38668f-e98c-4188-88d1-08fab5627a90","Type":"ContainerStarted","Data":"13bf013aee5f506803d2a30e7b3677dab42dc75b482ae42c9b2c7f84dd5a5a0a"}
Jul 24 16:23:10.347004 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:10.346864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8" event={"ID":"1a38668f-e98c-4188-88d1-08fab5627a90","Type":"ContainerStarted","Data":"a8a76a5ccc7da83317e9892c2f319fcf85bd2c4f18f521f0f620dde42c63e1ce"}
Jul 24 16:23:10.347004 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:10.346951    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:10.362957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:10.362909    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8" podStartSLOduration=1.362896053 podStartE2EDuration="1.362896053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:23:10.362136259 +0000 UTC m=+2172.689700807" watchObservedRunningTime="2026-07-24 16:23:10.362896053 +0000 UTC m=+2172.690460591"
Jul 24 16:23:13.188634 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:13.188582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:13.500797 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:13.500705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:13.804485 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:13.804459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:14.553369 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:14.553334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:16.664554 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:16.664503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:16.745202 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:16.745174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:16.822069 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:16.822039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:16.922892 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:16.922821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:17.317289 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.317259    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:23:17.406705 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.406674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:17.495304 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.495275    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:17.499541 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.499521    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"]
Jul 24 16:23:17.503831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.503818    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.505812 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.505792    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d060b0\""
Jul 24 16:23:17.512658 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.512633    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"]
Jul 24 16:23:17.542174 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.542146    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"]
Jul 24 16:23:17.545401 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.545383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.547685 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.547669    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d060b0-dockercfg-8rnkj\""
Jul 24 16:23:17.562824 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.562803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"]
Jul 24 16:23:17.631531 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.631468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a7fb8ce-3518-4280-8949-f27241ca6661-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.631531 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.631504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zkfj\" (UniqueName: \"kubernetes.io/projected/1a7fb8ce-3518-4280-8949-f27241ca6661-kube-api-access-8zkfj\") pod \"payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.732812 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.732785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbj4x\" (UniqueName: \"kubernetes.io/projected/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-kube-api-access-zbj4x\") pod \"payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.733179 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.732838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a7fb8ce-3518-4280-8949-f27241ca6661-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.733179 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.732900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zkfj\" (UniqueName: \"kubernetes.io/projected/1a7fb8ce-3518-4280-8949-f27241ca6661-kube-api-access-8zkfj\") pod \"payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.733179 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.732938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.733459 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.733443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a7fb8ce-3518-4280-8949-f27241ca6661-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.740755 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.740731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zkfj\" (UniqueName: \"kubernetes.io/projected/1a7fb8ce-3518-4280-8949-f27241ca6661-kube-api-access-8zkfj\") pod \"payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.817413 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.817388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:17.833721 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.833698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.833831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.833784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbj4x\" (UniqueName: \"kubernetes.io/projected/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-kube-api-access-zbj4x\") pod \"payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.834344 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.834323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.840956 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.840935    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbj4x\" (UniqueName: \"kubernetes.io/projected/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-kube-api-access-zbj4x\") pod \"payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.855963 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.855752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:17.957741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:17.957710    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"]
Jul 24 16:23:17.960239 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:23:17.960202    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7fb8ce_3518_4280_8949_f27241ca6661.slice/crio-6a192175ed90c2e7c0656414f9326909855a88b4713af9681c99e5268062a75e WatchSource:0}: Error finding container 6a192175ed90c2e7c0656414f9326909855a88b4713af9681c99e5268062a75e: Status 404 returned error can't find the container with id 6a192175ed90c2e7c0656414f9326909855a88b4713af9681c99e5268062a75e
Jul 24 16:23:18.005798 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.005775    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"]
Jul 24 16:23:18.008632 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:23:18.008591    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba93d8ea_de0d_4ff9_8e0e_f9eeeed38c54.slice/crio-35b776bb5a9d65b171458f0571aae6f0c5436aac4c8fae9dd88aa520a5da834a WatchSource:0}: Error finding container 35b776bb5a9d65b171458f0571aae6f0c5436aac4c8fae9dd88aa520a5da834a: Status 404 returned error can't find the container with id 35b776bb5a9d65b171458f0571aae6f0c5436aac4c8fae9dd88aa520a5da834a
Jul 24 16:23:18.388808 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.388769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454" event={"ID":"1a7fb8ce-3518-4280-8949-f27241ca6661","Type":"ContainerStarted","Data":"dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834"}
Jul 24 16:23:18.388986 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.388823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454" event={"ID":"1a7fb8ce-3518-4280-8949-f27241ca6661","Type":"ContainerStarted","Data":"6a192175ed90c2e7c0656414f9326909855a88b4713af9681c99e5268062a75e"}
Jul 24 16:23:18.388986 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.388860    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:18.390289 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.390270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n" event={"ID":"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54","Type":"ContainerStarted","Data":"c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83"}
Jul 24 16:23:18.390368 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.390310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:18.390368 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.390325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n" event={"ID":"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54","Type":"ContainerStarted","Data":"35b776bb5a9d65b171458f0571aae6f0c5436aac4c8fae9dd88aa520a5da834a"}
Jul 24 16:23:18.403251 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.403211    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454" podStartSLOduration=1.403196227 podStartE2EDuration="1.403196227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:23:18.402142044 +0000 UTC m=+2180.729706582" watchObservedRunningTime="2026-07-24 16:23:18.403196227 +0000 UTC m=+2180.730760766"
Jul 24 16:23:18.419268 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:18.419228    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n" podStartSLOduration=1.419217588 podStartE2EDuration="1.419217588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:23:18.417198361 +0000 UTC m=+2180.744762900" watchObservedRunningTime="2026-07-24 16:23:18.419217588 +0000 UTC m=+2180.746782188"
Jul 24 16:23:21.135841 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:21.135805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:21.353422 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:21.353400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:23:29.398021 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:29.397985    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:23:29.398482 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:29.398175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:23:32.097064 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:32.097031    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:23:44.788311 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.788268    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"]
Jul 24 16:23:44.791653 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.791637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.794108 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.794083    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e12d61\"/\"test-model-e12d61-kserve-self-signed-certs\""
Jul 24 16:23:44.794216 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.794156    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e12d61\"/\"openshift-service-ca.crt\""
Jul 24 16:23:44.794274 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.794227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e12d61\"/\"default-dockercfg-lsq57\""
Jul 24 16:23:44.794274 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.794235    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e12d61\"/\"kube-root-ca.crt\""
Jul 24 16:23:44.801909 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.801887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"]
Jul 24 16:23:44.824901 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.824876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-model-cache\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.824989 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.824912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-home\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.824989 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.824966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-tmp-dir\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.825067 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.824989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e9365-b7c8-4fb6-8522-ee1623968ba0-tls-certs\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.825067 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.825036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-dshm\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.825132 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.825092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl4r2\" (UniqueName: \"kubernetes.io/projected/965e9365-b7c8-4fb6-8522-ee1623968ba0-kube-api-access-rl4r2\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.896122 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.896092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:44.926339 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-model-cache\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926428 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-home\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926499 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-tmp-dir\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926541 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e9365-b7c8-4fb6-8522-ee1623968ba0-tls-certs\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926583 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-dshm\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926658 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl4r2\" (UniqueName: \"kubernetes.io/projected/965e9365-b7c8-4fb6-8522-ee1623968ba0-kube-api-access-rl4r2\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926761 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-model-cache\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.926831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.926784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-home\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.927109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.927085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-tmp-dir\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.931641 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.929648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-dshm\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.934936 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.934913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e9365-b7c8-4fb6-8522-ee1623968ba0-tls-certs\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:44.936687 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:44.936653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl4r2\" (UniqueName: \"kubernetes.io/projected/965e9365-b7c8-4fb6-8522-ee1623968ba0-kube-api-access-rl4r2\") pod \"test-model-e12d61-kserve-8499496cbc-zgl65\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") " pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:45.102303 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:45.102232    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:45.236909 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:45.236886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"]
Jul 24 16:23:45.239044 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:23:45.239019    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod965e9365_b7c8_4fb6_8522_ee1623968ba0.slice/crio-66a27d4521a1222f7a5b64dcbff3d0ed2a0d55d068df85213ea33c21ab9eb7d9 WatchSource:0}: Error finding container 66a27d4521a1222f7a5b64dcbff3d0ed2a0d55d068df85213ea33c21ab9eb7d9: Status 404 returned error can't find the container with id 66a27d4521a1222f7a5b64dcbff3d0ed2a0d55d068df85213ea33c21ab9eb7d9
Jul 24 16:23:45.542530 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:45.542494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65" event={"ID":"965e9365-b7c8-4fb6-8522-ee1623968ba0","Type":"ContainerStarted","Data":"7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2"}
Jul 24 16:23:45.542716 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:45.542542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65" event={"ID":"965e9365-b7c8-4fb6-8522-ee1623968ba0","Type":"ContainerStarted","Data":"66a27d4521a1222f7a5b64dcbff3d0ed2a0d55d068df85213ea33c21ab9eb7d9"}
Jul 24 16:23:45.560499 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:45.560456    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65" podStartSLOduration=1.560442104 podStartE2EDuration="1.560442104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:23:45.558525994 +0000 UTC m=+2207.886090532" watchObservedRunningTime="2026-07-24 16:23:45.560442104 +0000 UTC m=+2207.888006642"
Jul 24 16:23:47.793418 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:47.793380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:23:55.102892 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:55.102858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:55.103297 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:55.102919    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:55.115405 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:55.115380    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:55.607324 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:55.607295    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:23:58.471280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:58.471250    2577 scope.go:117] "RemoveContainer" containerID="963b722fba14ac94bab3d3ed0cf3cba442fb16f0a3c98bdfbf40871f10d9cd46"
Jul 24 16:23:59.295852 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:23:59.295822    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:24:27.430118 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:24:27.430043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:24:32.297907 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:24:32.297870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:25:07.821815 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:07.821768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:11.402462 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:11.402433    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"]
Jul 24 16:25:11.423840 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:11.423813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:12.269771 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.269743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:12.406403 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.406372    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"]
Jul 24 16:25:12.409911 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.409894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:12.411965 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.411935    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:25:12.411965 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.411961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:25:12.416978 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.416954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"]
Jul 24 16:25:12.529652 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.529537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b70283-1ce5-4f8f-a983-01ab6301a6e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv\" (UID: \"77b70283-1ce5-4f8f-a983-01ab6301a6e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:12.630376 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.630347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b70283-1ce5-4f8f-a983-01ab6301a6e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv\" (UID: \"77b70283-1ce5-4f8f-a983-01ab6301a6e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:12.631022 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.630998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b70283-1ce5-4f8f-a983-01ab6301a6e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv\" (UID: \"77b70283-1ce5-4f8f-a983-01ab6301a6e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:12.723910 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.723883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:12.852509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.852482    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"]
Jul 24 16:25:12.854904 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:25:12.854880    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b70283_1ce5_4f8f_a983_01ab6301a6e2.slice/crio-25154ce7253b62ca9ba936eabc6c5d17652c5207dd0b86b02590a81c89984943 WatchSource:0}: Error finding container 25154ce7253b62ca9ba936eabc6c5d17652c5207dd0b86b02590a81c89984943: Status 404 returned error can't find the container with id 25154ce7253b62ca9ba936eabc6c5d17652c5207dd0b86b02590a81c89984943
Jul 24 16:25:12.997815 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:12.997780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv" event={"ID":"77b70283-1ce5-4f8f-a983-01ab6301a6e2","Type":"ContainerStarted","Data":"25154ce7253b62ca9ba936eabc6c5d17652c5207dd0b86b02590a81c89984943"}
Jul 24 16:25:14.004585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:14.004497    2577 generic.go:356] "Generic (PLEG): container finished" podID="77b70283-1ce5-4f8f-a983-01ab6301a6e2" containerID="6b6b9586ba58ef6ea922f8751b4e241bc0e2833431eb53260c1fb2a62647c051" exitCode=0
Jul 24 16:25:14.004585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:14.004553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv" event={"ID":"77b70283-1ce5-4f8f-a983-01ab6301a6e2","Type":"ContainerDied","Data":"6b6b9586ba58ef6ea922f8751b4e241bc0e2833431eb53260c1fb2a62647c051"}
Jul 24 16:25:15.146662 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:15.146638    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:15.255679 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:15.255646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b70283-1ce5-4f8f-a983-01ab6301a6e2-maas-api-service-ca\") pod \"77b70283-1ce5-4f8f-a983-01ab6301a6e2\" (UID: \"77b70283-1ce5-4f8f-a983-01ab6301a6e2\") "
Jul 24 16:25:15.256038 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:15.256018    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b70283-1ce5-4f8f-a983-01ab6301a6e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77b70283-1ce5-4f8f-a983-01ab6301a6e2" (UID: "77b70283-1ce5-4f8f-a983-01ab6301a6e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:15.356970 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:15.356942    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b70283-1ce5-4f8f-a983-01ab6301a6e2-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.015936 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:16.015908    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"
Jul 24 16:25:16.016103 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:16.015910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv" event={"ID":"77b70283-1ce5-4f8f-a983-01ab6301a6e2","Type":"ContainerDied","Data":"25154ce7253b62ca9ba936eabc6c5d17652c5207dd0b86b02590a81c89984943"}
Jul 24 16:25:16.016103 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:16.016036    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25154ce7253b62ca9ba936eabc6c5d17652c5207dd0b86b02590a81c89984943"
Jul 24 16:25:16.487875 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:16.487842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:25:17.033954 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:17.033920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:17.132217 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:17.132182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:22.889428 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:22.889378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:22.931645 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:22.931601    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"]
Jul 24 16:25:22.931867 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:22.931822    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8" podUID="1a38668f-e98c-4188-88d1-08fab5627a90" containerName="payload-processing" containerID="cri-o://13bf013aee5f506803d2a30e7b3677dab42dc75b482ae42c9b2c7f84dd5a5a0a" gracePeriod=30
Jul 24 16:25:22.989555 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:22.989523    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:23.055876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.055845    2577 generic.go:356] "Generic (PLEG): container finished" podID="1a38668f-e98c-4188-88d1-08fab5627a90" containerID="13bf013aee5f506803d2a30e7b3677dab42dc75b482ae42c9b2c7f84dd5a5a0a" exitCode=0
Jul 24 16:25:23.055995 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.055922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8" event={"ID":"1a38668f-e98c-4188-88d1-08fab5627a90","Type":"ContainerDied","Data":"13bf013aee5f506803d2a30e7b3677dab42dc75b482ae42c9b2c7f84dd5a5a0a"}
Jul 24 16:25:23.090178 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.090148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:23.187581 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.187555    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:25:23.323467 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.323438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wcrs\" (UniqueName: \"kubernetes.io/projected/1a38668f-e98c-4188-88d1-08fab5627a90-kube-api-access-6wcrs\") pod \"1a38668f-e98c-4188-88d1-08fab5627a90\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") "
Jul 24 16:25:23.323673 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.323479    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a38668f-e98c-4188-88d1-08fab5627a90-plugins-config-volume\") pod \"1a38668f-e98c-4188-88d1-08fab5627a90\" (UID: \"1a38668f-e98c-4188-88d1-08fab5627a90\") "
Jul 24 16:25:23.323928 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.323903    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a38668f-e98c-4188-88d1-08fab5627a90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a38668f-e98c-4188-88d1-08fab5627a90" (UID: "1a38668f-e98c-4188-88d1-08fab5627a90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:23.325635 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.325602    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a38668f-e98c-4188-88d1-08fab5627a90-kube-api-access-6wcrs" (OuterVolumeSpecName: "kube-api-access-6wcrs") pod "1a38668f-e98c-4188-88d1-08fab5627a90" (UID: "1a38668f-e98c-4188-88d1-08fab5627a90"). InnerVolumeSpecName "kube-api-access-6wcrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:23.424775 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.424716    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wcrs\" (UniqueName: \"kubernetes.io/projected/1a38668f-e98c-4188-88d1-08fab5627a90-kube-api-access-6wcrs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:23.424775 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:23.424737    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a38668f-e98c-4188-88d1-08fab5627a90-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:24.062432 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:24.062402    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"
Jul 24 16:25:24.062432 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:24.062419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8" event={"ID":"1a38668f-e98c-4188-88d1-08fab5627a90","Type":"ContainerDied","Data":"a8a76a5ccc7da83317e9892c2f319fcf85bd2c4f18f521f0f620dde42c63e1ce"}
Jul 24 16:25:24.063003 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:24.062509    2577 scope.go:117] "RemoveContainer" containerID="13bf013aee5f506803d2a30e7b3677dab42dc75b482ae42c9b2c7f84dd5a5a0a"
Jul 24 16:25:24.084698 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:24.084678    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"]
Jul 24 16:25:24.088354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:24.088334    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e12d61-7f84d9b8bd-2nmv8"]
Jul 24 16:25:24.239114 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:24.239083    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a38668f-e98c-4188-88d1-08fab5627a90" path="/var/lib/kubelet/pods/1a38668f-e98c-4188-88d1-08fab5627a90/volumes"
Jul 24 16:25:26.421357 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.421304    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65" podUID="965e9365-b7c8-4fb6-8522-ee1623968ba0" containerName="main" containerID="cri-o://7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2" gracePeriod=45
Jul 24 16:25:26.662887 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.662866    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:25:26.751163 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751100    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-model-cache\") pod \"965e9365-b7c8-4fb6-8522-ee1623968ba0\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") "
Jul 24 16:25:26.751163 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751137    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rl4r2\" (UniqueName: \"kubernetes.io/projected/965e9365-b7c8-4fb6-8522-ee1623968ba0-kube-api-access-rl4r2\") pod \"965e9365-b7c8-4fb6-8522-ee1623968ba0\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") "
Jul 24 16:25:26.751163 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751159    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e9365-b7c8-4fb6-8522-ee1623968ba0-tls-certs\") pod \"965e9365-b7c8-4fb6-8522-ee1623968ba0\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") "
Jul 24 16:25:26.751409 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751200    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-home\") pod \"965e9365-b7c8-4fb6-8522-ee1623968ba0\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") "
Jul 24 16:25:26.751409 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751237    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-tmp-dir\") pod \"965e9365-b7c8-4fb6-8522-ee1623968ba0\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") "
Jul 24 16:25:26.751409 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751259    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-dshm\") pod \"965e9365-b7c8-4fb6-8522-ee1623968ba0\" (UID: \"965e9365-b7c8-4fb6-8522-ee1623968ba0\") "
Jul 24 16:25:26.751559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751399    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-model-cache" (OuterVolumeSpecName: "model-cache") pod "965e9365-b7c8-4fb6-8522-ee1623968ba0" (UID: "965e9365-b7c8-4fb6-8522-ee1623968ba0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:26.751559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751504    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:26.751559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751505    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "965e9365-b7c8-4fb6-8522-ee1623968ba0" (UID: "965e9365-b7c8-4fb6-8522-ee1623968ba0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:26.751559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.751513    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-home" (OuterVolumeSpecName: "home") pod "965e9365-b7c8-4fb6-8522-ee1623968ba0" (UID: "965e9365-b7c8-4fb6-8522-ee1623968ba0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:26.753331 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.753308    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-dshm" (OuterVolumeSpecName: "dshm") pod "965e9365-b7c8-4fb6-8522-ee1623968ba0" (UID: "965e9365-b7c8-4fb6-8522-ee1623968ba0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:26.753331 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.753326    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/965e9365-b7c8-4fb6-8522-ee1623968ba0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "965e9365-b7c8-4fb6-8522-ee1623968ba0" (UID: "965e9365-b7c8-4fb6-8522-ee1623968ba0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:25:26.753484 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.753360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965e9365-b7c8-4fb6-8522-ee1623968ba0-kube-api-access-rl4r2" (OuterVolumeSpecName: "kube-api-access-rl4r2") pod "965e9365-b7c8-4fb6-8522-ee1623968ba0" (UID: "965e9365-b7c8-4fb6-8522-ee1623968ba0"). InnerVolumeSpecName "kube-api-access-rl4r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:26.851897 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.851867    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl4r2\" (UniqueName: \"kubernetes.io/projected/965e9365-b7c8-4fb6-8522-ee1623968ba0-kube-api-access-rl4r2\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:26.851897 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.851890    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e9365-b7c8-4fb6-8522-ee1623968ba0-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:26.852031 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.851905    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:26.852031 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.851914    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:26.852031 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:26.851921    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e9365-b7c8-4fb6-8522-ee1623968ba0-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:27.081474 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.081447    2577 generic.go:356] "Generic (PLEG): container finished" podID="965e9365-b7c8-4fb6-8522-ee1623968ba0" containerID="7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2" exitCode=0
Jul 24 16:25:27.081593 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.081488    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"
Jul 24 16:25:27.081593 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.081532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65" event={"ID":"965e9365-b7c8-4fb6-8522-ee1623968ba0","Type":"ContainerDied","Data":"7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2"}
Jul 24 16:25:27.081593 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.081567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65" event={"ID":"965e9365-b7c8-4fb6-8522-ee1623968ba0","Type":"ContainerDied","Data":"66a27d4521a1222f7a5b64dcbff3d0ed2a0d55d068df85213ea33c21ab9eb7d9"}
Jul 24 16:25:27.081593 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.081582    2577 scope.go:117] "RemoveContainer" containerID="7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2"
Jul 24 16:25:27.090709 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.090689    2577 scope.go:117] "RemoveContainer" containerID="7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2"
Jul 24 16:25:27.090978 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:25:27.090957    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2\": container with ID starting with 7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2 not found: ID does not exist" containerID="7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2"
Jul 24 16:25:27.091049 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.090987    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2"} err="failed to get container status \"7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2\": rpc error: code = NotFound desc = could not find container \"7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2\": container with ID starting with 7710241babc0187f7137b4292730f441c20ed0cb3d77920ea1a25d43181460b2 not found: ID does not exist"
Jul 24 16:25:27.103749 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.103728    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"]
Jul 24 16:25:27.108785 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.108766    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e12d61/test-model-e12d61-kserve-8499496cbc-zgl65"]
Jul 24 16:25:27.482219 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.482160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:27.498274 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.498249    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"]
Jul 24 16:25:27.501060 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.501039    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e12d61-djhnv"]
Jul 24 16:25:27.869795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:27.869765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:28.243578 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:28.243499    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b70283-1ce5-4f8f-a983-01ab6301a6e2" path="/var/lib/kubelet/pods/77b70283-1ce5-4f8f-a983-01ab6301a6e2/volumes"
Jul 24 16:25:28.243895 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:28.243881    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="965e9365-b7c8-4fb6-8522-ee1623968ba0" path="/var/lib/kubelet/pods/965e9365-b7c8-4fb6-8522-ee1623968ba0/volumes"
Jul 24 16:25:32.491792 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:32.491760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:25:33.656177 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:33.656140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:33.783642 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:33.783593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:33.965523 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:33.965440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:34.203525 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.203500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:34.308440 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308413    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"]
Jul 24 16:25:34.308760 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308747    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a38668f-e98c-4188-88d1-08fab5627a90" containerName="payload-processing"
Jul 24 16:25:34.308806 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308763    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a38668f-e98c-4188-88d1-08fab5627a90" containerName="payload-processing"
Jul 24 16:25:34.308806 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308775    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="965e9365-b7c8-4fb6-8522-ee1623968ba0" containerName="main"
Jul 24 16:25:34.308806 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308781    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="965e9365-b7c8-4fb6-8522-ee1623968ba0" containerName="main"
Jul 24 16:25:34.308806 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308793    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b70283-1ce5-4f8f-a983-01ab6301a6e2" containerName="revoke-keys"
Jul 24 16:25:34.308806 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308799    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b70283-1ce5-4f8f-a983-01ab6301a6e2" containerName="revoke-keys"
Jul 24 16:25:34.308973 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308869    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a38668f-e98c-4188-88d1-08fab5627a90" containerName="payload-processing"
Jul 24 16:25:34.308973 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308878    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b70283-1ce5-4f8f-a983-01ab6301a6e2" containerName="revoke-keys"
Jul 24 16:25:34.308973 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.308884    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="965e9365-b7c8-4fb6-8522-ee1623968ba0" containerName="main"
Jul 24 16:25:34.311945 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.311924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:34.313752 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.313731    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:25:34.313871 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.313856    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:25:34.318665 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.318646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"]
Jul 24 16:25:34.410715 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.410690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr\" (UID: \"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:34.512028 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.512007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr\" (UID: \"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:34.512552 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.512536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr\" (UID: \"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:34.622805 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.622756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:34.751786 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:34.751764    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"]
Jul 24 16:25:34.753790 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:25:34.753755    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeed48b4_d3e9_4e22_9237_9ef3d0eb7d11.slice/crio-24611786e5c7552c7fd4114696be6fb383e97a71d549a3cb4f4158047cf67e72 WatchSource:0}: Error finding container 24611786e5c7552c7fd4114696be6fb383e97a71d549a3cb4f4158047cf67e72: Status 404 returned error can't find the container with id 24611786e5c7552c7fd4114696be6fb383e97a71d549a3cb4f4158047cf67e72
Jul 24 16:25:35.126880 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:35.126850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr" event={"ID":"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11","Type":"ContainerStarted","Data":"f8cfec873385d170e890c6f410171949e4c4f5bf36b62a6e5f67dd9e42ba620e"}
Jul 24 16:25:35.126993 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:35.126897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr" event={"ID":"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11","Type":"ContainerStarted","Data":"24611786e5c7552c7fd4114696be6fb383e97a71d549a3cb4f4158047cf67e72"}
Jul 24 16:25:35.143406 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:35.143194    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr" podStartSLOduration=1.14317707 podStartE2EDuration="1.14317707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:25:35.142688424 +0000 UTC m=+2317.470252962" watchObservedRunningTime="2026-07-24 16:25:35.14317707 +0000 UTC m=+2317.470741610"
Jul 24 16:25:36.133121 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:36.133089    2577 generic.go:356] "Generic (PLEG): container finished" podID="eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11" containerID="f8cfec873385d170e890c6f410171949e4c4f5bf36b62a6e5f67dd9e42ba620e" exitCode=0
Jul 24 16:25:36.133479 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:36.133173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr" event={"ID":"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11","Type":"ContainerDied","Data":"f8cfec873385d170e890c6f410171949e4c4f5bf36b62a6e5f67dd9e42ba620e"}
Jul 24 16:25:37.269343 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:37.269322    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:37.329673 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:37.329652    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11-maas-api-service-ca\") pod \"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11\" (UID: \"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11\") "
Jul 24 16:25:37.329991 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:37.329962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11" (UID: "eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:37.430489 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:37.430431    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:38.144271 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:38.144245    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"
Jul 24 16:25:38.144271 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:38.144262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr" event={"ID":"eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11","Type":"ContainerDied","Data":"24611786e5c7552c7fd4114696be6fb383e97a71d549a3cb4f4158047cf67e72"}
Jul 24 16:25:38.144502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:38.144329    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24611786e5c7552c7fd4114696be6fb383e97a71d549a3cb4f4158047cf67e72"
Jul 24 16:25:44.808242 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:44.808211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:44.821798 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:44.821768    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"]
Jul 24 16:25:44.822009 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:44.821987    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n" podUID="ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" containerName="payload-processing" containerID="cri-o://c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83" gracePeriod=30
Jul 24 16:25:44.830342 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:44.830319    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"]
Jul 24 16:25:44.830849 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:44.830529    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454" podUID="1a7fb8ce-3518-4280-8949-f27241ca6661" containerName="payload-pre-processing" containerID="cri-o://dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834" gracePeriod=30
Jul 24 16:25:44.906012 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:44.905984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:45.082941 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.082912    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:25:45.106189 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.106165    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:25:45.186875 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.186845    2577 generic.go:356] "Generic (PLEG): container finished" podID="ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" containerID="c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83" exitCode=0
Jul 24 16:25:45.187023 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.186889    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"
Jul 24 16:25:45.187023 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.186934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n" event={"ID":"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54","Type":"ContainerDied","Data":"c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83"}
Jul 24 16:25:45.187023 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.186985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n" event={"ID":"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54","Type":"ContainerDied","Data":"35b776bb5a9d65b171458f0571aae6f0c5436aac4c8fae9dd88aa520a5da834a"}
Jul 24 16:25:45.187023 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.187013    2577 scope.go:117] "RemoveContainer" containerID="c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83"
Jul 24 16:25:45.188247 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188216    2577 generic.go:356] "Generic (PLEG): container finished" podID="1a7fb8ce-3518-4280-8949-f27241ca6661" containerID="dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834" exitCode=0
Jul 24 16:25:45.188361 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188255    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"
Jul 24 16:25:45.188361 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454" event={"ID":"1a7fb8ce-3518-4280-8949-f27241ca6661","Type":"ContainerDied","Data":"dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834"}
Jul 24 16:25:45.188361 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188296    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbj4x\" (UniqueName: \"kubernetes.io/projected/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-kube-api-access-zbj4x\") pod \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") "
Jul 24 16:25:45.188361 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454" event={"ID":"1a7fb8ce-3518-4280-8949-f27241ca6661","Type":"ContainerDied","Data":"6a192175ed90c2e7c0656414f9326909855a88b4713af9681c99e5268062a75e"}
Jul 24 16:25:45.188361 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188331    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-plugins-config-volume\") pod \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\" (UID: \"ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54\") "
Jul 24 16:25:45.188574 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zkfj\" (UniqueName: \"kubernetes.io/projected/1a7fb8ce-3518-4280-8949-f27241ca6661-kube-api-access-8zkfj\") pod \"1a7fb8ce-3518-4280-8949-f27241ca6661\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") "
Jul 24 16:25:45.188574 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a7fb8ce-3518-4280-8949-f27241ca6661-plugins-config-volume\") pod \"1a7fb8ce-3518-4280-8949-f27241ca6661\" (UID: \"1a7fb8ce-3518-4280-8949-f27241ca6661\") "
Jul 24 16:25:45.188888 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188853    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" (UID: "ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:45.188969 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.188897    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a7fb8ce-3518-4280-8949-f27241ca6661-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a7fb8ce-3518-4280-8949-f27241ca6661" (UID: "1a7fb8ce-3518-4280-8949-f27241ca6661"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:45.190464 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.190443    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7fb8ce-3518-4280-8949-f27241ca6661-kube-api-access-8zkfj" (OuterVolumeSpecName: "kube-api-access-8zkfj") pod "1a7fb8ce-3518-4280-8949-f27241ca6661" (UID: "1a7fb8ce-3518-4280-8949-f27241ca6661"). InnerVolumeSpecName "kube-api-access-8zkfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:45.190464 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.190456    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-kube-api-access-zbj4x" (OuterVolumeSpecName: "kube-api-access-zbj4x") pod "ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" (UID: "ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54"). InnerVolumeSpecName "kube-api-access-zbj4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:45.195743 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.195726    2577 scope.go:117] "RemoveContainer" containerID="c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83"
Jul 24 16:25:45.195974 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:25:45.195955    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83\": container with ID starting with c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83 not found: ID does not exist" containerID="c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83"
Jul 24 16:25:45.196032 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.195981    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83"} err="failed to get container status \"c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83\": rpc error: code = NotFound desc = could not find container \"c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83\": container with ID starting with c9ff5de568814726c729e68b16b058234450a79f42e7240e1adf69f67663fb83 not found: ID does not exist"
Jul 24 16:25:45.196032 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.195996    2577 scope.go:117] "RemoveContainer" containerID="dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834"
Jul 24 16:25:45.203478 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.203463    2577 scope.go:117] "RemoveContainer" containerID="dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834"
Jul 24 16:25:45.203785 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:25:45.203768    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834\": container with ID starting with dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834 not found: ID does not exist" containerID="dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834"
Jul 24 16:25:45.203855 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.203791    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834"} err="failed to get container status \"dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834\": rpc error: code = NotFound desc = could not find container \"dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834\": container with ID starting with dd264ef9ff47b37071c23b78c487d2739ba5ac8425889f8f096079493c7aa834 not found: ID does not exist"
Jul 24 16:25:45.289109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.289087    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbj4x\" (UniqueName: \"kubernetes.io/projected/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-kube-api-access-zbj4x\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:45.289201 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.289111    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:45.289201 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.289125    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zkfj\" (UniqueName: \"kubernetes.io/projected/1a7fb8ce-3518-4280-8949-f27241ca6661-kube-api-access-8zkfj\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:45.289201 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.289137    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a7fb8ce-3518-4280-8949-f27241ca6661-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:25:45.509746 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.509724    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"]
Jul 24 16:25:45.513177 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.513156    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d060b0-7957947d5b-7vs5n"]
Jul 24 16:25:45.522846 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.522815    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"]
Jul 24 16:25:45.527564 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:45.527538    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d060b0-6c554d9b76-df454"]
Jul 24 16:25:46.237716 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:46.237644    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a7fb8ce-3518-4280-8949-f27241ca6661" path="/var/lib/kubelet/pods/1a7fb8ce-3518-4280-8949-f27241ca6661/volumes"
Jul 24 16:25:46.238042 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:46.238011    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" path="/var/lib/kubelet/pods/ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54/volumes"
Jul 24 16:25:47.793724 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:47.793692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:25:49.392942 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:49.392907    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"]
Jul 24 16:25:49.395008 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:49.394980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:49.395351 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:49.395329    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d060b0-s9wlr"]
Jul 24 16:25:49.769600 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:49.769527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:50.238815 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:50.238783    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11" path="/var/lib/kubelet/pods/eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11/volumes"
Jul 24 16:25:55.716225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:55.716195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:55.817382 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:55.817356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:55.988902 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:55.988819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:56.219197 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:56.219163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:56.583540 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:56.583508    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:56.906218 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:56.906140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:57.087387 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:57.087358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:58.498098 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:58.498072    2577 scope.go:117] "RemoveContainer" containerID="a5eea94b5822a8c9eed4988bd54fd7e4f41f32cccf7a239919f8ed2fbb19e5a0"
Jul 24 16:25:59.605914 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:59.605886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:59.706135 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:59.706105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:59.812063 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:25:59.812032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:00.043939 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.043912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:00.406183 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.406114    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:00.524932 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.524899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:00.604571 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.604535    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"]
Jul 24 16:26:00.605124 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605105    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11" containerName="revoke-keys"
Jul 24 16:26:00.605176 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605128    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11" containerName="revoke-keys"
Jul 24 16:26:00.605176 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605145    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7fb8ce-3518-4280-8949-f27241ca6661" containerName="payload-pre-processing"
Jul 24 16:26:00.605176 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605153    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7fb8ce-3518-4280-8949-f27241ca6661" containerName="payload-pre-processing"
Jul 24 16:26:00.605270 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605195    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" containerName="payload-processing"
Jul 24 16:26:00.605270 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605203    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" containerName="payload-processing"
Jul 24 16:26:00.605337 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605281    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba93d8ea-de0d-4ff9-8e0e-f9eeeed38c54" containerName="payload-processing"
Jul 24 16:26:00.605337 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605298    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a7fb8ce-3518-4280-8949-f27241ca6661" containerName="payload-pre-processing"
Jul 24 16:26:00.605337 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.605307    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeed48b4-d3e9-4e22-9237-9ef3d0eb7d11" containerName="revoke-keys"
Jul 24 16:26:00.610204 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.610182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.612629 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.612598    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-01d58c\""
Jul 24 16:26:00.624386 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.624361    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"]
Jul 24 16:26:00.695899 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.695822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74p9f\" (UniqueName: \"kubernetes.io/projected/95d9572a-4cef-409c-9e9c-34790f1bbcfe-kube-api-access-74p9f\") pod \"payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.695899 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.695880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d9572a-4cef-409c-9e9c-34790f1bbcfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.743407 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.743379    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"]
Jul 24 16:26:00.746751 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.746736    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:00.748737 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.748714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-01d58c-dockercfg-t75np\""
Jul 24 16:26:00.756582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.756559    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"]
Jul 24 16:26:00.796733 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.796705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d9572a-4cef-409c-9e9c-34790f1bbcfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.796871 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.796769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74p9f\" (UniqueName: \"kubernetes.io/projected/95d9572a-4cef-409c-9e9c-34790f1bbcfe-kube-api-access-74p9f\") pod \"payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.797417 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.797392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d9572a-4cef-409c-9e9c-34790f1bbcfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.804147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.804118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74p9f\" (UniqueName: \"kubernetes.io/projected/95d9572a-4cef-409c-9e9c-34790f1bbcfe-kube-api-access-74p9f\") pod \"payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.897410 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.897369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:00.897577 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.897432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xbp\" (UniqueName: \"kubernetes.io/projected/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-kube-api-access-v4xbp\") pod \"payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:00.920355 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.920327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:00.998651 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.998600    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xbp\" (UniqueName: \"kubernetes.io/projected/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-kube-api-access-v4xbp\") pod \"payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:00.998819 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.998751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:00.999460 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:00.999441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:01.007004 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.006981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xbp\" (UniqueName: \"kubernetes.io/projected/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-kube-api-access-v4xbp\") pod \"payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:01.057123 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.057080    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:01.057702 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.057679    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"]
Jul 24 16:26:01.059499 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:26:01.059470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d9572a_4cef_409c_9e9c_34790f1bbcfe.slice/crio-7a9f6488bf994a0b8b06adf495be64314f324e3432fc2823753b17a726ed5eac WatchSource:0}: Error finding container 7a9f6488bf994a0b8b06adf495be64314f324e3432fc2823753b17a726ed5eac: Status 404 returned error can't find the container with id 7a9f6488bf994a0b8b06adf495be64314f324e3432fc2823753b17a726ed5eac
Jul 24 16:26:01.191112 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.191090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"]
Jul 24 16:26:01.192780 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:26:01.192757    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448a63ec_5db4_49ca_b34e_b60c11d2e2b7.slice/crio-434ccb7e79756638414a76f37e9d7ecf1997b5b3cf69adc1b1b424361db97875 WatchSource:0}: Error finding container 434ccb7e79756638414a76f37e9d7ecf1997b5b3cf69adc1b1b424361db97875: Status 404 returned error can't find the container with id 434ccb7e79756638414a76f37e9d7ecf1997b5b3cf69adc1b1b424361db97875
Jul 24 16:26:01.279015 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.278985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk" event={"ID":"448a63ec-5db4-49ca-b34e-b60c11d2e2b7","Type":"ContainerStarted","Data":"cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae"}
Jul 24 16:26:01.279198 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.279181    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:01.279263 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.279223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk" event={"ID":"448a63ec-5db4-49ca-b34e-b60c11d2e2b7","Type":"ContainerStarted","Data":"434ccb7e79756638414a76f37e9d7ecf1997b5b3cf69adc1b1b424361db97875"}
Jul 24 16:26:01.280726 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.280699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54" event={"ID":"95d9572a-4cef-409c-9e9c-34790f1bbcfe","Type":"ContainerStarted","Data":"ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978"}
Jul 24 16:26:01.280876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.280737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54" event={"ID":"95d9572a-4cef-409c-9e9c-34790f1bbcfe","Type":"ContainerStarted","Data":"7a9f6488bf994a0b8b06adf495be64314f324e3432fc2823753b17a726ed5eac"}
Jul 24 16:26:01.280876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.280834    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:01.300870 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.300809    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk" podStartSLOduration=1.300795485 podStartE2EDuration="1.300795485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:01.297124507 +0000 UTC m=+2343.624689066" watchObservedRunningTime="2026-07-24 16:26:01.300795485 +0000 UTC m=+2343.628360076"
Jul 24 16:26:01.318082 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:01.318034    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54" podStartSLOduration=1.3180196149999999 podStartE2EDuration="1.318019615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:01.316403941 +0000 UTC m=+2343.643968497" watchObservedRunningTime="2026-07-24 16:26:01.318019615 +0000 UTC m=+2343.645584166"
Jul 24 16:26:04.120799 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:04.120766    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:04.597870 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:04.597835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:04.935507 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:04.935430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:05.308644 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:05.308602    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:26:05.361034 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:05.361004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:07.715764 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:07.715727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:07.853836 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:07.853800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:08.244672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.244637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:08.445758 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.445711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:08.531928 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.531898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:08.537097 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.537074    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"]
Jul 24 16:26:08.540583 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.540568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.542352 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.542332    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2c74ca\""
Jul 24 16:26:08.550688 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.550669    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"]
Jul 24 16:26:08.583562 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.583538    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"]
Jul 24 16:26:08.586874 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.586860    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.588664 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.588646    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2c74ca-dockercfg-8x8t9\""
Jul 24 16:26:08.597208 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.597187    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"]
Jul 24 16:26:08.664707 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.664676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a4a3ad-13b1-442d-88e3-2710ae930c43-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.664876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.664720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsbbl\" (UniqueName: \"kubernetes.io/projected/79a4a3ad-13b1-442d-88e3-2710ae930c43-kube-api-access-hsbbl\") pod \"payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.765774 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.765740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmz8s\" (UniqueName: \"kubernetes.io/projected/4615fc59-4517-4912-8cb0-956f92b01048-kube-api-access-xmz8s\") pod \"payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") " pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.766228 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.765783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4615fc59-4517-4912-8cb0-956f92b01048-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") " pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.766228 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.765814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a4a3ad-13b1-442d-88e3-2710ae930c43-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.766228 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.765870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsbbl\" (UniqueName: \"kubernetes.io/projected/79a4a3ad-13b1-442d-88e3-2710ae930c43-kube-api-access-hsbbl\") pod \"payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.766500 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.766480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a4a3ad-13b1-442d-88e3-2710ae930c43-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.775024 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.775000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsbbl\" (UniqueName: \"kubernetes.io/projected/79a4a3ad-13b1-442d-88e3-2710ae930c43-kube-api-access-hsbbl\") pod \"payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.852225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.852158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:08.866676 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.866650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmz8s\" (UniqueName: \"kubernetes.io/projected/4615fc59-4517-4912-8cb0-956f92b01048-kube-api-access-xmz8s\") pod \"payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") " pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.866776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.866710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4615fc59-4517-4912-8cb0-956f92b01048-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") " pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.867409 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.867388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4615fc59-4517-4912-8cb0-956f92b01048-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") " pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.874558 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.874181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmz8s\" (UniqueName: \"kubernetes.io/projected/4615fc59-4517-4912-8cb0-956f92b01048-kube-api-access-xmz8s\") pod \"payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") " pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.897177 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.897143    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:08.990404 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:08.990375    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"]
Jul 24 16:26:08.992233 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:26:08.992207    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a4a3ad_13b1_442d_88e3_2710ae930c43.slice/crio-2a29333adafce16c3059f6215a008b7f5b521b0dedda610d95b63b9b0e4dbac8 WatchSource:0}: Error finding container 2a29333adafce16c3059f6215a008b7f5b521b0dedda610d95b63b9b0e4dbac8: Status 404 returned error can't find the container with id 2a29333adafce16c3059f6215a008b7f5b521b0dedda610d95b63b9b0e4dbac8
Jul 24 16:26:09.047402 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.047376    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"]
Jul 24 16:26:09.049771 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:26:09.049750    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4615fc59_4517_4912_8cb0_956f92b01048.slice/crio-5f534c3f155cff0770ceb4b9429640e28beaaae056473fdb7bffa24d3dc9a0fd WatchSource:0}: Error finding container 5f534c3f155cff0770ceb4b9429640e28beaaae056473fdb7bffa24d3dc9a0fd: Status 404 returned error can't find the container with id 5f534c3f155cff0770ceb4b9429640e28beaaae056473fdb7bffa24d3dc9a0fd
Jul 24 16:26:09.323921 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.323890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp" event={"ID":"79a4a3ad-13b1-442d-88e3-2710ae930c43","Type":"ContainerStarted","Data":"38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb"}
Jul 24 16:26:09.324106 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.323992    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:09.324106 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.324021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp" event={"ID":"79a4a3ad-13b1-442d-88e3-2710ae930c43","Type":"ContainerStarted","Data":"2a29333adafce16c3059f6215a008b7f5b521b0dedda610d95b63b9b0e4dbac8"}
Jul 24 16:26:09.325503 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.325479    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh" event={"ID":"4615fc59-4517-4912-8cb0-956f92b01048","Type":"ContainerStarted","Data":"425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab"}
Jul 24 16:26:09.325598 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.325522    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh" event={"ID":"4615fc59-4517-4912-8cb0-956f92b01048","Type":"ContainerStarted","Data":"5f534c3f155cff0770ceb4b9429640e28beaaae056473fdb7bffa24d3dc9a0fd"}
Jul 24 16:26:09.325598 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.325549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:09.338804 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.338762    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp" podStartSLOduration=1.338748126 podStartE2EDuration="1.338748126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:09.337564817 +0000 UTC m=+2351.665129376" watchObservedRunningTime="2026-07-24 16:26:09.338748126 +0000 UTC m=+2351.666312664"
Jul 24 16:26:09.352934 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:09.352894    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh" podStartSLOduration=1.352879963 podStartE2EDuration="1.352879963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:09.351887923 +0000 UTC m=+2351.679452462" watchObservedRunningTime="2026-07-24 16:26:09.352879963 +0000 UTC m=+2351.680444495"
Jul 24 16:26:12.160199 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:12.160162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:12.288192 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:12.288164    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:12.288328 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:12.288225    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:20.231308 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.231269    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"]
Jul 24 16:26:20.235103 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.235081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:20.236906 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.236881    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:26:20.237028 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.236925    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:26:20.244437 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.244414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"]
Jul 24 16:26:20.270353 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.270325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c3709c0-b1df-4f18-a22b-3908fa26422d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-01d58c-t77p5\" (UID: \"1c3709c0-b1df-4f18-a22b-3908fa26422d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:20.334550 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.334528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:26:20.335243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.335226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:26:20.371738 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.371711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c3709c0-b1df-4f18-a22b-3908fa26422d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-01d58c-t77p5\" (UID: \"1c3709c0-b1df-4f18-a22b-3908fa26422d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:20.372680 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.372661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c3709c0-b1df-4f18-a22b-3908fa26422d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-01d58c-t77p5\" (UID: \"1c3709c0-b1df-4f18-a22b-3908fa26422d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:20.551202 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.551176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:20.678900 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:20.678872    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"]
Jul 24 16:26:20.680332 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:26:20.680308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3709c0_b1df_4f18_a22b_3908fa26422d.slice/crio-3260ca419f17ee58bb4dfa6d591d436560dbc4fda9312e84c2f98bb80e489286 WatchSource:0}: Error finding container 3260ca419f17ee58bb4dfa6d591d436560dbc4fda9312e84c2f98bb80e489286: Status 404 returned error can't find the container with id 3260ca419f17ee58bb4dfa6d591d436560dbc4fda9312e84c2f98bb80e489286
Jul 24 16:26:21.391541 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:21.391502    2577 generic.go:356] "Generic (PLEG): container finished" podID="1c3709c0-b1df-4f18-a22b-3908fa26422d" containerID="285513c69d4502d4c19e146b38cbf8df79d471aa8b345933006192a775f69cc3" exitCode=0
Jul 24 16:26:21.391913 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:21.391595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5" event={"ID":"1c3709c0-b1df-4f18-a22b-3908fa26422d","Type":"ContainerDied","Data":"285513c69d4502d4c19e146b38cbf8df79d471aa8b345933006192a775f69cc3"}
Jul 24 16:26:21.391913 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:21.391679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5" event={"ID":"1c3709c0-b1df-4f18-a22b-3908fa26422d","Type":"ContainerStarted","Data":"3260ca419f17ee58bb4dfa6d591d436560dbc4fda9312e84c2f98bb80e489286"}
Jul 24 16:26:22.522739 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:22.522717    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:22.589168 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:22.589145    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c3709c0-b1df-4f18-a22b-3908fa26422d-maas-api-service-ca\") pod \"1c3709c0-b1df-4f18-a22b-3908fa26422d\" (UID: \"1c3709c0-b1df-4f18-a22b-3908fa26422d\") "
Jul 24 16:26:22.589518 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:22.589498    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3709c0-b1df-4f18-a22b-3908fa26422d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c3709c0-b1df-4f18-a22b-3908fa26422d" (UID: "1c3709c0-b1df-4f18-a22b-3908fa26422d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:26:22.690004 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:22.689930    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c3709c0-b1df-4f18-a22b-3908fa26422d-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:26:22.905504 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:22.905475    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:26:23.404077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:23.404046    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"
Jul 24 16:26:23.404077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:23.404072    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5" event={"ID":"1c3709c0-b1df-4f18-a22b-3908fa26422d","Type":"ContainerDied","Data":"3260ca419f17ee58bb4dfa6d591d436560dbc4fda9312e84c2f98bb80e489286"}
Jul 24 16:26:23.404345 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:23.404115    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3260ca419f17ee58bb4dfa6d591d436560dbc4fda9312e84c2f98bb80e489286"
Jul 24 16:26:29.505410 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:29.505372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:36.067180 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.067141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:36.093710 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.093682    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"]
Jul 24 16:26:36.093934 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.093900    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk" podUID="448a63ec-5db4-49ca-b34e-b60c11d2e2b7" containerName="payload-processing" containerID="cri-o://cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae" gracePeriod=30
Jul 24 16:26:36.106044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.106016    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"]
Jul 24 16:26:36.106280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.106258    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54" podUID="95d9572a-4cef-409c-9e9c-34790f1bbcfe" containerName="payload-pre-processing" containerID="cri-o://ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978" gracePeriod=30
Jul 24 16:26:36.135081 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.135058    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:36.196576 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.196551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:36.371811 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.371789    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:36.375362 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.375343    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:36.473474 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.473442    2577 generic.go:356] "Generic (PLEG): container finished" podID="95d9572a-4cef-409c-9e9c-34790f1bbcfe" containerID="ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978" exitCode=0
Jul 24 16:26:36.473675 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.473484    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"
Jul 24 16:26:36.473675 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.473537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54" event={"ID":"95d9572a-4cef-409c-9e9c-34790f1bbcfe","Type":"ContainerDied","Data":"ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978"}
Jul 24 16:26:36.473675 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.473597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54" event={"ID":"95d9572a-4cef-409c-9e9c-34790f1bbcfe","Type":"ContainerDied","Data":"7a9f6488bf994a0b8b06adf495be64314f324e3432fc2823753b17a726ed5eac"}
Jul 24 16:26:36.473675 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.473643    2577 scope.go:117] "RemoveContainer" containerID="ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978"
Jul 24 16:26:36.474852 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.474830    2577 generic.go:356] "Generic (PLEG): container finished" podID="448a63ec-5db4-49ca-b34e-b60c11d2e2b7" containerID="cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae" exitCode=0
Jul 24 16:26:36.474957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.474868    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"
Jul 24 16:26:36.474957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.474873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk" event={"ID":"448a63ec-5db4-49ca-b34e-b60c11d2e2b7","Type":"ContainerDied","Data":"cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae"}
Jul 24 16:26:36.474957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.474910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk" event={"ID":"448a63ec-5db4-49ca-b34e-b60c11d2e2b7","Type":"ContainerDied","Data":"434ccb7e79756638414a76f37e9d7ecf1997b5b3cf69adc1b1b424361db97875"}
Jul 24 16:26:36.482274 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.482257    2577 scope.go:117] "RemoveContainer" containerID="ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978"
Jul 24 16:26:36.482511 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:26:36.482491    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978\": container with ID starting with ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978 not found: ID does not exist" containerID="ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978"
Jul 24 16:26:36.482558 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.482519    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978"} err="failed to get container status \"ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978\": rpc error: code = NotFound desc = could not find container \"ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978\": container with ID starting with ad49f417d4074cb889a3d203ce50bb0d2da1ee7c25af9638bdd93ec7ade74978 not found: ID does not exist"
Jul 24 16:26:36.482558 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.482534    2577 scope.go:117] "RemoveContainer" containerID="cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae"
Jul 24 16:26:36.490001 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.489987    2577 scope.go:117] "RemoveContainer" containerID="cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae"
Jul 24 16:26:36.490228 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:26:36.490214    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae\": container with ID starting with cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae not found: ID does not exist" containerID="cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae"
Jul 24 16:26:36.490270 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.490235    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae"} err="failed to get container status \"cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae\": rpc error: code = NotFound desc = could not find container \"cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae\": container with ID starting with cf37813b8859b0611d701d08a4162d8c3e070e9362fcd071f1d5f598a27ddaae not found: ID does not exist"
Jul 24 16:26:36.494569 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.494551    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74p9f\" (UniqueName: \"kubernetes.io/projected/95d9572a-4cef-409c-9e9c-34790f1bbcfe-kube-api-access-74p9f\") pod \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") "
Jul 24 16:26:36.494641 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.494582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d9572a-4cef-409c-9e9c-34790f1bbcfe-plugins-config-volume\") pod \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\" (UID: \"95d9572a-4cef-409c-9e9c-34790f1bbcfe\") "
Jul 24 16:26:36.494641 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.494603    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-plugins-config-volume\") pod \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") "
Jul 24 16:26:36.494757 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.494645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4xbp\" (UniqueName: \"kubernetes.io/projected/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-kube-api-access-v4xbp\") pod \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\" (UID: \"448a63ec-5db4-49ca-b34e-b60c11d2e2b7\") "
Jul 24 16:26:36.494937 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.494917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "448a63ec-5db4-49ca-b34e-b60c11d2e2b7" (UID: "448a63ec-5db4-49ca-b34e-b60c11d2e2b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:26:36.495038 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.495015    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d9572a-4cef-409c-9e9c-34790f1bbcfe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95d9572a-4cef-409c-9e9c-34790f1bbcfe" (UID: "95d9572a-4cef-409c-9e9c-34790f1bbcfe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:26:36.496469 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.496450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d9572a-4cef-409c-9e9c-34790f1bbcfe-kube-api-access-74p9f" (OuterVolumeSpecName: "kube-api-access-74p9f") pod "95d9572a-4cef-409c-9e9c-34790f1bbcfe" (UID: "95d9572a-4cef-409c-9e9c-34790f1bbcfe"). InnerVolumeSpecName "kube-api-access-74p9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:26:36.496552 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.496535    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-kube-api-access-v4xbp" (OuterVolumeSpecName: "kube-api-access-v4xbp") pod "448a63ec-5db4-49ca-b34e-b60c11d2e2b7" (UID: "448a63ec-5db4-49ca-b34e-b60c11d2e2b7"). InnerVolumeSpecName "kube-api-access-v4xbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:26:36.595563 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.595537    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74p9f\" (UniqueName: \"kubernetes.io/projected/95d9572a-4cef-409c-9e9c-34790f1bbcfe-kube-api-access-74p9f\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:26:36.595563 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.595559    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d9572a-4cef-409c-9e9c-34790f1bbcfe-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:26:36.595744 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.595570    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:26:36.595744 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.595583    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4xbp\" (UniqueName: \"kubernetes.io/projected/448a63ec-5db4-49ca-b34e-b60c11d2e2b7-kube-api-access-v4xbp\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:26:36.797366 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.797339    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"]
Jul 24 16:26:36.800531 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.800505    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-01d58c-78f5bd8974-wjzpk"]
Jul 24 16:26:36.816307 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.816280    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"]
Jul 24 16:26:36.818329 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:36.818309    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-01d58c-7b9b57cc68-tdc54"]
Jul 24 16:26:38.237754 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:38.237723    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="448a63ec-5db4-49ca-b34e-b60c11d2e2b7" path="/var/lib/kubelet/pods/448a63ec-5db4-49ca-b34e-b60c11d2e2b7/volumes"
Jul 24 16:26:38.238274 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:38.238199    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d9572a-4cef-409c-9e9c-34790f1bbcfe" path="/var/lib/kubelet/pods/95d9572a-4cef-409c-9e9c-34790f1bbcfe/volumes"
Jul 24 16:26:40.319264 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:40.319235    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"]
Jul 24 16:26:40.321485 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:40.321455    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01d58c-t77p5"]
Jul 24 16:26:40.322003 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:40.321978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:40.681394 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:40.681316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:40.889551 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:40.889520    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:26:42.238184 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:42.238150    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c3709c0-b1df-4f18-a22b-3908fa26422d" path="/var/lib/kubelet/pods/1c3709c0-b1df-4f18-a22b-3908fa26422d/volumes"
Jul 24 16:26:46.838748 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:46.838715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:46.940057 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:46.940019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:47.140705 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.140589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:47.364845 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.364806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:47.487832 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.487755    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"]
Jul 24 16:26:47.488171 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488147    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="448a63ec-5db4-49ca-b34e-b60c11d2e2b7" containerName="payload-processing"
Jul 24 16:26:47.488171 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488169    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="448a63ec-5db4-49ca-b34e-b60c11d2e2b7" containerName="payload-processing"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488187    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d9572a-4cef-409c-9e9c-34790f1bbcfe" containerName="payload-pre-processing"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488192    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d9572a-4cef-409c-9e9c-34790f1bbcfe" containerName="payload-pre-processing"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488201    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c3709c0-b1df-4f18-a22b-3908fa26422d" containerName="revoke-keys"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488206    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c3709c0-b1df-4f18-a22b-3908fa26422d" containerName="revoke-keys"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488270    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c3709c0-b1df-4f18-a22b-3908fa26422d" containerName="revoke-keys"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488283    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="448a63ec-5db4-49ca-b34e-b60c11d2e2b7" containerName="payload-processing"
Jul 24 16:26:47.488338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.488290    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d9572a-4cef-409c-9e9c-34790f1bbcfe" containerName="payload-pre-processing"
Jul 24 16:26:47.492426 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.492411    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:47.494452 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.494433    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:26:47.494559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.494453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:26:47.498879 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.498857    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"]
Jul 24 16:26:47.581734 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.581704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d368c688-681a-44a3-8a05-16f65a32208e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl\" (UID: \"d368c688-681a-44a3-8a05-16f65a32208e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:47.682177 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.682140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d368c688-681a-44a3-8a05-16f65a32208e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl\" (UID: \"d368c688-681a-44a3-8a05-16f65a32208e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:47.682775 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.682756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d368c688-681a-44a3-8a05-16f65a32208e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl\" (UID: \"d368c688-681a-44a3-8a05-16f65a32208e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:47.805503 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.805463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:47.937330 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:47.937305    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"]
Jul 24 16:26:47.939875 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:26:47.939840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd368c688_681a_44a3_8a05_16f65a32208e.slice/crio-c0cccfd78d033fbbc858eecbde5ee4eb965aeb9b05b97669bdc5957cfab6df68 WatchSource:0}: Error finding container c0cccfd78d033fbbc858eecbde5ee4eb965aeb9b05b97669bdc5957cfab6df68: Status 404 returned error can't find the container with id c0cccfd78d033fbbc858eecbde5ee4eb965aeb9b05b97669bdc5957cfab6df68
Jul 24 16:26:48.540008 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:48.539979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl" event={"ID":"d368c688-681a-44a3-8a05-16f65a32208e","Type":"ContainerStarted","Data":"939e5460b3f9096c0c5b43b091d5224b5480876a33f1334b59cb3face49d8508"}
Jul 24 16:26:48.540175 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:48.540029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl" event={"ID":"d368c688-681a-44a3-8a05-16f65a32208e","Type":"ContainerStarted","Data":"c0cccfd78d033fbbc858eecbde5ee4eb965aeb9b05b97669bdc5957cfab6df68"}
Jul 24 16:26:48.554342 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:48.554300    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl" podStartSLOduration=1.55428572 podStartE2EDuration="1.55428572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:48.553426617 +0000 UTC m=+2390.880991167" watchObservedRunningTime="2026-07-24 16:26:48.55428572 +0000 UTC m=+2390.881850257"
Jul 24 16:26:49.547068 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:49.547034    2577 generic.go:356] "Generic (PLEG): container finished" podID="d368c688-681a-44a3-8a05-16f65a32208e" containerID="939e5460b3f9096c0c5b43b091d5224b5480876a33f1334b59cb3face49d8508" exitCode=0
Jul 24 16:26:49.547512 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:49.547120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl" event={"ID":"d368c688-681a-44a3-8a05-16f65a32208e","Type":"ContainerDied","Data":"939e5460b3f9096c0c5b43b091d5224b5480876a33f1334b59cb3face49d8508"}
Jul 24 16:26:50.680518 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:50.680491    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:50.808719 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:50.808642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d368c688-681a-44a3-8a05-16f65a32208e-maas-api-service-ca\") pod \"d368c688-681a-44a3-8a05-16f65a32208e\" (UID: \"d368c688-681a-44a3-8a05-16f65a32208e\") "
Jul 24 16:26:50.808964 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:50.808945    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d368c688-681a-44a3-8a05-16f65a32208e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d368c688-681a-44a3-8a05-16f65a32208e" (UID: "d368c688-681a-44a3-8a05-16f65a32208e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:26:50.909447 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:50.909418    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d368c688-681a-44a3-8a05-16f65a32208e-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:26:51.558648 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:51.558605    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"
Jul 24 16:26:51.558648 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:51.558641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl" event={"ID":"d368c688-681a-44a3-8a05-16f65a32208e","Type":"ContainerDied","Data":"c0cccfd78d033fbbc858eecbde5ee4eb965aeb9b05b97669bdc5957cfab6df68"}
Jul 24 16:26:51.558855 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:26:51.558687    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0cccfd78d033fbbc858eecbde5ee4eb965aeb9b05b97669bdc5957cfab6df68"
Jul 24 16:27:02.936860 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:02.936818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:02.964230 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:02.964204    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"]
Jul 24 16:27:02.964444 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:02.964423    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh" podUID="4615fc59-4517-4912-8cb0-956f92b01048" containerName="payload-processing" containerID="cri-o://425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab" gracePeriod=30
Jul 24 16:27:02.972651 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:02.972624    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"]
Jul 24 16:27:02.972876 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:02.972820    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp" podUID="79a4a3ad-13b1-442d-88e3-2710ae930c43" containerName="payload-pre-processing" containerID="cri-o://38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb" gracePeriod=30
Jul 24 16:27:03.009265 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.009234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:03.112786 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.112754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:03.226993 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.226967    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:27:03.243559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.243534    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:27:03.308958 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.308932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4615fc59-4517-4912-8cb0-956f92b01048-plugins-config-volume\") pod \"4615fc59-4517-4912-8cb0-956f92b01048\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") "
Jul 24 16:27:03.309082 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.308988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsbbl\" (UniqueName: \"kubernetes.io/projected/79a4a3ad-13b1-442d-88e3-2710ae930c43-kube-api-access-hsbbl\") pod \"79a4a3ad-13b1-442d-88e3-2710ae930c43\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") "
Jul 24 16:27:03.309082 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.309015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a4a3ad-13b1-442d-88e3-2710ae930c43-plugins-config-volume\") pod \"79a4a3ad-13b1-442d-88e3-2710ae930c43\" (UID: \"79a4a3ad-13b1-442d-88e3-2710ae930c43\") "
Jul 24 16:27:03.309223 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.309201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmz8s\" (UniqueName: \"kubernetes.io/projected/4615fc59-4517-4912-8cb0-956f92b01048-kube-api-access-xmz8s\") pod \"4615fc59-4517-4912-8cb0-956f92b01048\" (UID: \"4615fc59-4517-4912-8cb0-956f92b01048\") "
Jul 24 16:27:03.309293 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.309267    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4615fc59-4517-4912-8cb0-956f92b01048-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4615fc59-4517-4912-8cb0-956f92b01048" (UID: "4615fc59-4517-4912-8cb0-956f92b01048"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:27:03.309338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.309275    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79a4a3ad-13b1-442d-88e3-2710ae930c43-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79a4a3ad-13b1-442d-88e3-2710ae930c43" (UID: "79a4a3ad-13b1-442d-88e3-2710ae930c43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:27:03.309483 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.309466    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4615fc59-4517-4912-8cb0-956f92b01048-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:27:03.309542 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.309484    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a4a3ad-13b1-442d-88e3-2710ae930c43-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:27:03.311070 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.311046    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4615fc59-4517-4912-8cb0-956f92b01048-kube-api-access-xmz8s" (OuterVolumeSpecName: "kube-api-access-xmz8s") pod "4615fc59-4517-4912-8cb0-956f92b01048" (UID: "4615fc59-4517-4912-8cb0-956f92b01048"). InnerVolumeSpecName "kube-api-access-xmz8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:27:03.311158 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.311087    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a4a3ad-13b1-442d-88e3-2710ae930c43-kube-api-access-hsbbl" (OuterVolumeSpecName: "kube-api-access-hsbbl") pod "79a4a3ad-13b1-442d-88e3-2710ae930c43" (UID: "79a4a3ad-13b1-442d-88e3-2710ae930c43"). InnerVolumeSpecName "kube-api-access-hsbbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:27:03.410147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.410115    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsbbl\" (UniqueName: \"kubernetes.io/projected/79a4a3ad-13b1-442d-88e3-2710ae930c43-kube-api-access-hsbbl\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:27:03.410147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.410146    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmz8s\" (UniqueName: \"kubernetes.io/projected/4615fc59-4517-4912-8cb0-956f92b01048-kube-api-access-xmz8s\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:27:03.628620 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.628590    2577 generic.go:356] "Generic (PLEG): container finished" podID="79a4a3ad-13b1-442d-88e3-2710ae930c43" containerID="38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb" exitCode=0
Jul 24 16:27:03.628776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.628650    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"
Jul 24 16:27:03.628776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.628680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp" event={"ID":"79a4a3ad-13b1-442d-88e3-2710ae930c43","Type":"ContainerDied","Data":"38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb"}
Jul 24 16:27:03.628776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.628735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp" event={"ID":"79a4a3ad-13b1-442d-88e3-2710ae930c43","Type":"ContainerDied","Data":"2a29333adafce16c3059f6215a008b7f5b521b0dedda610d95b63b9b0e4dbac8"}
Jul 24 16:27:03.628776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.628761    2577 scope.go:117] "RemoveContainer" containerID="38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb"
Jul 24 16:27:03.630045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.630024    2577 generic.go:356] "Generic (PLEG): container finished" podID="4615fc59-4517-4912-8cb0-956f92b01048" containerID="425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab" exitCode=0
Jul 24 16:27:03.630162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.630064    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"
Jul 24 16:27:03.630162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.630069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh" event={"ID":"4615fc59-4517-4912-8cb0-956f92b01048","Type":"ContainerDied","Data":"425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab"}
Jul 24 16:27:03.630162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.630103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh" event={"ID":"4615fc59-4517-4912-8cb0-956f92b01048","Type":"ContainerDied","Data":"5f534c3f155cff0770ceb4b9429640e28beaaae056473fdb7bffa24d3dc9a0fd"}
Jul 24 16:27:03.638127 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.637954    2577 scope.go:117] "RemoveContainer" containerID="38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb"
Jul 24 16:27:03.638280 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:27:03.638254    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb\": container with ID starting with 38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb not found: ID does not exist" containerID="38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb"
Jul 24 16:27:03.638335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.638293    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb"} err="failed to get container status \"38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb\": rpc error: code = NotFound desc = could not find container \"38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb\": container with ID starting with 38c66186226dd8c26979406d62b060a8d05b575b8e7fbc0027af764c91fc45bb not found: ID does not exist"
Jul 24 16:27:03.638335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.638316    2577 scope.go:117] "RemoveContainer" containerID="425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab"
Jul 24 16:27:03.646151 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.646138    2577 scope.go:117] "RemoveContainer" containerID="425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab"
Jul 24 16:27:03.646371 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:27:03.646355    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab\": container with ID starting with 425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab not found: ID does not exist" containerID="425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab"
Jul 24 16:27:03.646420 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.646377    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab"} err="failed to get container status \"425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab\": rpc error: code = NotFound desc = could not find container \"425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab\": container with ID starting with 425324c13c0c757a4eb72518de5f76d4db874b3c9f1efa4ee1642dbb88056eab not found: ID does not exist"
Jul 24 16:27:03.652310 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.652290    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"]
Jul 24 16:27:03.657935 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.657915    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2c74ca-5bfc88f6d6-kqkjh"]
Jul 24 16:27:03.666567 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.666548    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"]
Jul 24 16:27:03.670337 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:03.670319    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2c74ca-f6478dd-4tvkp"]
Jul 24 16:27:04.238771 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:04.238699    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4615fc59-4517-4912-8cb0-956f92b01048" path="/var/lib/kubelet/pods/4615fc59-4517-4912-8cb0-956f92b01048/volumes"
Jul 24 16:27:04.239107 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:04.239050    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79a4a3ad-13b1-442d-88e3-2710ae930c43" path="/var/lib/kubelet/pods/79a4a3ad-13b1-442d-88e3-2710ae930c43/volumes"
Jul 24 16:27:07.566919 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:07.566890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:07.567311 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:07.567074    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"]
Jul 24 16:27:07.569927 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:07.569902    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2c74ca-jf2nl"]
Jul 24 16:27:07.951914 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:07.951838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:08.190370 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:08.190337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:27:08.238629 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:08.238551    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d368c688-681a-44a3-8a05-16f65a32208e" path="/var/lib/kubelet/pods/d368c688-681a-44a3-8a05-16f65a32208e/volumes"
Jul 24 16:27:13.861906 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:13.861867    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:13.962492 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:13.962453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:14.130128 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:14.130048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:14.357737 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:14.357706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:14.689600 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:14.689571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:15.023939 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:15.023911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:15.206033 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:15.206003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:17.985708 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:17.985676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:18.090047 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.090017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:18.221000 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.220969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:18.463348 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.463318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:18.695367 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.695333    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:18.783216 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.783187    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:18.790979 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.790950    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"]
Jul 24 16:27:18.791305 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791288    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a4a3ad-13b1-442d-88e3-2710ae930c43" containerName="payload-pre-processing"
Jul 24 16:27:18.791383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791307    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a4a3ad-13b1-442d-88e3-2710ae930c43" containerName="payload-pre-processing"
Jul 24 16:27:18.791383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791339    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fc59-4517-4912-8cb0-956f92b01048" containerName="payload-processing"
Jul 24 16:27:18.791383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791349    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fc59-4517-4912-8cb0-956f92b01048" containerName="payload-processing"
Jul 24 16:27:18.791383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791369    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d368c688-681a-44a3-8a05-16f65a32208e" containerName="revoke-keys"
Jul 24 16:27:18.791383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791377    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d368c688-681a-44a3-8a05-16f65a32208e" containerName="revoke-keys"
Jul 24 16:27:18.791668 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791461    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="79a4a3ad-13b1-442d-88e3-2710ae930c43" containerName="payload-pre-processing"
Jul 24 16:27:18.791668 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791474    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d368c688-681a-44a3-8a05-16f65a32208e" containerName="revoke-keys"
Jul 24 16:27:18.791668 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.791484    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fc59-4517-4912-8cb0-956f92b01048" containerName="payload-processing"
Jul 24 16:27:18.794703 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.794686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:18.797402 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.797371    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-a5fa08-dockercfg-4877l\""
Jul 24 16:27:18.797506 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.797436    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-a5fa08\""
Jul 24 16:27:18.807283 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.807261    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"]
Jul 24 16:27:18.828567 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.826706    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"]
Jul 24 16:27:18.831452 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.831429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:18.836096 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.836066    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"]
Jul 24 16:27:18.931585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.931556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5118b6c4-b0b4-4dd6-8b40-d9814a279402-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:18.931755 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.931591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d83819a0-3789-4143-844a-9b09aee4cdbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:18.931755 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.931609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q44t2\" (UniqueName: \"kubernetes.io/projected/d83819a0-3789-4143-844a-9b09aee4cdbf-kube-api-access-q44t2\") pod \"payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:18.931838 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:18.931751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfk96\" (UniqueName: \"kubernetes.io/projected/5118b6c4-b0b4-4dd6-8b40-d9814a279402-kube-api-access-vfk96\") pod \"payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.032473 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.032437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5118b6c4-b0b4-4dd6-8b40-d9814a279402-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.032473 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.032476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d83819a0-3789-4143-844a-9b09aee4cdbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:19.032915 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.032494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q44t2\" (UniqueName: \"kubernetes.io/projected/d83819a0-3789-4143-844a-9b09aee4cdbf-kube-api-access-q44t2\") pod \"payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:19.032915 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.032556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfk96\" (UniqueName: \"kubernetes.io/projected/5118b6c4-b0b4-4dd6-8b40-d9814a279402-kube-api-access-vfk96\") pod \"payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.033146 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.033127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d83819a0-3789-4143-844a-9b09aee4cdbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:19.033247 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.033206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5118b6c4-b0b4-4dd6-8b40-d9814a279402-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.040357 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.040336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q44t2\" (UniqueName: \"kubernetes.io/projected/d83819a0-3789-4143-844a-9b09aee4cdbf-kube-api-access-q44t2\") pod \"payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:19.040473 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.040366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfk96\" (UniqueName: \"kubernetes.io/projected/5118b6c4-b0b4-4dd6-8b40-d9814a279402-kube-api-access-vfk96\") pod \"payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.104464 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.104435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:19.143413 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.143386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.245972 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.245947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"]
Jul 24 16:27:19.248113 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:27:19.248070    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83819a0_3789_4143_844a_9b09aee4cdbf.slice/crio-1f1533bcc536090ccff68a5cfe891b5bfeb3b7e8edf572eeb68b3d4781d083af WatchSource:0}: Error finding container 1f1533bcc536090ccff68a5cfe891b5bfeb3b7e8edf572eeb68b3d4781d083af: Status 404 returned error can't find the container with id 1f1533bcc536090ccff68a5cfe891b5bfeb3b7e8edf572eeb68b3d4781d083af
Jul 24 16:27:19.292466 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.292441    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"]
Jul 24 16:27:19.292876 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:27:19.292855    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5118b6c4_b0b4_4dd6_8b40_d9814a279402.slice/crio-b8ed1e1d80b4321bafea19087436d355ed1ce10b29f929fe25907c42751d5427 WatchSource:0}: Error finding container b8ed1e1d80b4321bafea19087436d355ed1ce10b29f929fe25907c42751d5427: Status 404 returned error can't find the container with id b8ed1e1d80b4321bafea19087436d355ed1ce10b29f929fe25907c42751d5427
Jul 24 16:27:19.720248 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.720159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd" event={"ID":"d83819a0-3789-4143-844a-9b09aee4cdbf","Type":"ContainerStarted","Data":"f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286"}
Jul 24 16:27:19.720248 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.720214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd" event={"ID":"d83819a0-3789-4143-844a-9b09aee4cdbf","Type":"ContainerStarted","Data":"1f1533bcc536090ccff68a5cfe891b5bfeb3b7e8edf572eeb68b3d4781d083af"}
Jul 24 16:27:19.720451 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.720327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:19.721695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.721672    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l" event={"ID":"5118b6c4-b0b4-4dd6-8b40-d9814a279402","Type":"ContainerStarted","Data":"24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b"}
Jul 24 16:27:19.721818 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.721708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l" event={"ID":"5118b6c4-b0b4-4dd6-8b40-d9814a279402","Type":"ContainerStarted","Data":"b8ed1e1d80b4321bafea19087436d355ed1ce10b29f929fe25907c42751d5427"}
Jul 24 16:27:19.721818 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.721732    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:19.735831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.735788    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd" podStartSLOduration=1.7357743220000001 podStartE2EDuration="1.735774322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:27:19.733668684 +0000 UTC m=+2422.061233242" watchObservedRunningTime="2026-07-24 16:27:19.735774322 +0000 UTC m=+2422.063338885"
Jul 24 16:27:19.750086 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:19.750048    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l" podStartSLOduration=1.7500355189999999 podStartE2EDuration="1.750035519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:27:19.748597347 +0000 UTC m=+2422.076161910" watchObservedRunningTime="2026-07-24 16:27:19.750035519 +0000 UTC m=+2422.077600057"
Jul 24 16:27:22.289420 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:22.289393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:27:22.495505 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:22.495455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:22.807694 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:22.807667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:23.121213 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:23.121138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:23.595857 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:23.595827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.044127 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.044089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.124200 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.124169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.209800 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.209771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.566673 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.566649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.723268 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.723239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.822414 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.822340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:26.845125 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.845103    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"]
Jul 24 16:27:26.849730 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.849712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:26.851750 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.851734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-675edf-dockercfg-245kl\""
Jul 24 16:27:26.851857 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.851735    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-675edf\""
Jul 24 16:27:26.858363 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.858344    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"]
Jul 24 16:27:26.861944 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.861927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"]
Jul 24 16:27:26.862047 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.862006    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:26.869825 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.869803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"]
Jul 24 16:27:26.999708 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.999676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkkdv\" (UniqueName: \"kubernetes.io/projected/e98bba96-c386-49a2-b340-df57d150a72f-kube-api-access-gkkdv\") pod \"payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:26.999866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.999715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49e908d-8827-4f36-86f6-b02333476718-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") " pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:26.999866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.999779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2t7v\" (UniqueName: \"kubernetes.io/projected/d49e908d-8827-4f36-86f6-b02333476718-kube-api-access-g2t7v\") pod \"payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") " pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:26.999866 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:26.999821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e98bba96-c386-49a2-b340-df57d150a72f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.100607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.100536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkkdv\" (UniqueName: \"kubernetes.io/projected/e98bba96-c386-49a2-b340-df57d150a72f-kube-api-access-gkkdv\") pod \"payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.100607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.100572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49e908d-8827-4f36-86f6-b02333476718-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") " pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:27.100607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.100602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2t7v\" (UniqueName: \"kubernetes.io/projected/d49e908d-8827-4f36-86f6-b02333476718-kube-api-access-g2t7v\") pod \"payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") " pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:27.101117 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.100650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e98bba96-c386-49a2-b340-df57d150a72f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.101317 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.101298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49e908d-8827-4f36-86f6-b02333476718-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") " pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:27.101363 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.101349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e98bba96-c386-49a2-b340-df57d150a72f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.107961 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.107931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkkdv\" (UniqueName: \"kubernetes.io/projected/e98bba96-c386-49a2-b340-df57d150a72f-kube-api-access-gkkdv\") pod \"payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.108084 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.107980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2t7v\" (UniqueName: \"kubernetes.io/projected/d49e908d-8827-4f36-86f6-b02333476718-kube-api-access-g2t7v\") pod \"payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") " pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:27.160692 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.160671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.172776 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.172760    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:27.301081 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.301054    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"]
Jul 24 16:27:27.303352 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:27:27.303323    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98bba96_c386_49a2_b340_df57d150a72f.slice/crio-cfb0ee562f1080f915fe636055909f9d30235bdb1e124d91d48540e57be74a25 WatchSource:0}: Error finding container cfb0ee562f1080f915fe636055909f9d30235bdb1e124d91d48540e57be74a25: Status 404 returned error can't find the container with id cfb0ee562f1080f915fe636055909f9d30235bdb1e124d91d48540e57be74a25
Jul 24 16:27:27.530007 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.529981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"]
Jul 24 16:27:27.531536 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:27:27.531504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49e908d_8827_4f36_86f6_b02333476718.slice/crio-5086393c2d24e29e267c999716d03ef443ae4351980118cbd0fa7287036aa13c WatchSource:0}: Error finding container 5086393c2d24e29e267c999716d03ef443ae4351980118cbd0fa7287036aa13c: Status 404 returned error can't find the container with id 5086393c2d24e29e267c999716d03ef443ae4351980118cbd0fa7287036aa13c
Jul 24 16:27:27.767659 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.767568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" event={"ID":"e98bba96-c386-49a2-b340-df57d150a72f","Type":"ContainerStarted","Data":"89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c"}
Jul 24 16:27:27.767659 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.767647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" event={"ID":"e98bba96-c386-49a2-b340-df57d150a72f","Type":"ContainerStarted","Data":"cfb0ee562f1080f915fe636055909f9d30235bdb1e124d91d48540e57be74a25"}
Jul 24 16:27:27.767860 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.767684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:27.769293 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.769271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh" event={"ID":"d49e908d-8827-4f36-86f6-b02333476718","Type":"ContainerStarted","Data":"f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c"}
Jul 24 16:27:27.769409 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.769314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh" event={"ID":"d49e908d-8827-4f36-86f6-b02333476718","Type":"ContainerStarted","Data":"5086393c2d24e29e267c999716d03ef443ae4351980118cbd0fa7287036aa13c"}
Jul 24 16:27:27.769409 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.769358    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:27.781532 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.781484    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" podStartSLOduration=1.7814685529999998 podStartE2EDuration="1.781468553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:27:27.781122123 +0000 UTC m=+2430.108686673" watchObservedRunningTime="2026-07-24 16:27:27.781468553 +0000 UTC m=+2430.109033094"
Jul 24 16:27:27.796098 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:27.796062    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh" podStartSLOduration=1.7960501359999999 podStartE2EDuration="1.796050136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:27:27.794796177 +0000 UTC m=+2430.122360739" watchObservedRunningTime="2026-07-24 16:27:27.796050136 +0000 UTC m=+2430.123614673"
Jul 24 16:27:30.506817 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:30.506785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:30.729577 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:30.729545    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:27:30.729748 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:30.729717    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:27:36.798192 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:36.798154    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:27:38.776483 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:38.776438    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" podUID="e98bba96-c386-49a2-b340-df57d150a72f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.72:9004: connect: connection refused"
Jul 24 16:27:38.776962 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:38.776752    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:27:39.718014 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:39.717983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:48.775951 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:48.775920    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:27:53.287480 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:27:53.287449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:28:03.604140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.604103    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"]
Jul 24 16:28:03.608926 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.608909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.611268 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.611245    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"openshift-service-ca.crt\""
Jul 24 16:28:03.611373 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.611250    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"kube-root-ca.crt\""
Jul 24 16:28:03.611373 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.611289    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"default-dockercfg-rb79d\""
Jul 24 16:28:03.611373 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.611271    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"auth-test-model-675edf-kserve-self-signed-certs\""
Jul 24 16:28:03.618560 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.618540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"]
Jul 24 16:28:03.676518 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.676488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-dshm\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.676700 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.676539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr7zn\" (UniqueName: \"kubernetes.io/projected/de89db4a-5bee-4477-90b0-201efe550bfd-kube-api-access-qr7zn\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.676759 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.676715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de89db4a-5bee-4477-90b0-201efe550bfd-tls-certs\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.676817 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.676793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-tmp-dir\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.676873 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.676846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-home\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.676926 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.676871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-model-cache\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.703884 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.703860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:03.777784 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.777756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de89db4a-5bee-4477-90b0-201efe550bfd-tls-certs\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.777950 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.777801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-tmp-dir\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.777950 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.777847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-home\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.777950 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.777869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-model-cache\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.777950 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.777897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-dshm\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.777950 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.777930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr7zn\" (UniqueName: \"kubernetes.io/projected/de89db4a-5bee-4477-90b0-201efe550bfd-kube-api-access-qr7zn\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.778302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.778280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-home\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.778382 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.778314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-tmp-dir\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.778382 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.778356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-model-cache\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.780185 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.780156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-dshm\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.780309 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.780294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de89db4a-5bee-4477-90b0-201efe550bfd-tls-certs\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.787524 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.787500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr7zn\" (UniqueName: \"kubernetes.io/projected/de89db4a-5bee-4477-90b0-201efe550bfd-kube-api-access-qr7zn\") pod \"auth-test-model-675edf-kserve-6f844d989f-p8btl\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") " pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:03.918669 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:03.918562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:04.059307 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:04.059277    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"]
Jul 24 16:28:04.061888 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:28:04.061849    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde89db4a_5bee_4477_90b0_201efe550bfd.slice/crio-00eb0d3a5ab84b79bb207df229429315a8ae9d5070b423e6479eff42f30661db WatchSource:0}: Error finding container 00eb0d3a5ab84b79bb207df229429315a8ae9d5070b423e6479eff42f30661db: Status 404 returned error can't find the container with id 00eb0d3a5ab84b79bb207df229429315a8ae9d5070b423e6479eff42f30661db
Jul 24 16:28:04.969162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:04.969121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl" event={"ID":"de89db4a-5bee-4477-90b0-201efe550bfd","Type":"ContainerStarted","Data":"77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3"}
Jul 24 16:28:04.969162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:04.969164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl" event={"ID":"de89db4a-5bee-4477-90b0-201efe550bfd","Type":"ContainerStarted","Data":"00eb0d3a5ab84b79bb207df229429315a8ae9d5070b423e6479eff42f30661db"}
Jul 24 16:28:04.986231 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:04.986026    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl" podStartSLOduration=1.9860118839999998 podStartE2EDuration="1.986011884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:28:04.985497881 +0000 UTC m=+2467.313062420" watchObservedRunningTime="2026-07-24 16:28:04.986011884 +0000 UTC m=+2467.313576425"
Jul 24 16:28:07.488288 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:07.488254    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:28:13.918829 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:13.918796    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:13.919418 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:13.918849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:13.931475 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:13.931449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:14.026458 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:14.026430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:28:38.192996 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:28:38.192963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:29:04.685157 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:29:04.685074    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:29:21.705414 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:29:21.705379    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:29:38.338056 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:29:38.338022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:49.289742 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:29:49.289711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:30:00.136367 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.136311    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"]
Jul 24 16:30:00.140512 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.140490    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"]
Jul 24 16:30:00.140674 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.140626    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"
Jul 24 16:30:00.142425 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.142406    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:30:00.144057 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.144040    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"]
Jul 24 16:30:00.144158 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.144141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"
Jul 24 16:30:00.147249 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.147233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"
Jul 24 16:30:00.152321 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.152301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"
Jul 24 16:30:00.154361 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.154271    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"]
Jul 24 16:30:00.160025 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.158601    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"]
Jul 24 16:30:00.162233 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.161279    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"]
Jul 24 16:30:00.165455 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.165422    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"
Jul 24 16:30:00.203656 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.203634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"
Jul 24 16:30:00.370466 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.370440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"]
Jul 24 16:30:00.372020 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:30:00.371999    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80737204_2412_4455_9b04_e4bdf5d0843e.slice/crio-9041cdeb34cac4cdf23a27c9460241a310d3dfa3c8da241bc84e846128e41060 WatchSource:0}: Error finding container 9041cdeb34cac4cdf23a27c9460241a310d3dfa3c8da241bc84e846128e41060: Status 404 returned error can't find the container with id 9041cdeb34cac4cdf23a27c9460241a310d3dfa3c8da241bc84e846128e41060
Jul 24 16:30:00.520677 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.520647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"]
Jul 24 16:30:00.521715 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:30:00.521687    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86bd3c6_d5b0_4928_bffb_f4ddbc13abf5.slice/crio-c1e2983a0b2420d1da7f23366da073f590ed9a06e31db4716916dca5e7960f6e WatchSource:0}: Error finding container c1e2983a0b2420d1da7f23366da073f590ed9a06e31db4716916dca5e7960f6e: Status 404 returned error can't find the container with id c1e2983a0b2420d1da7f23366da073f590ed9a06e31db4716916dca5e7960f6e
Jul 24 16:30:00.549655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.549604    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"]
Jul 24 16:30:00.551769 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:30:00.551741    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91985fc2_3a73_4d4a_8eb4_3fbbe0ff6c41.slice/crio-dcbf8798b0dca7881b665214996888c2e2b712aa9d2e1be96581e243f7b2f868 WatchSource:0}: Error finding container dcbf8798b0dca7881b665214996888c2e2b712aa9d2e1be96581e243f7b2f868: Status 404 returned error can't find the container with id dcbf8798b0dca7881b665214996888c2e2b712aa9d2e1be96581e243f7b2f868
Jul 24 16:30:00.591727 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.591700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq" event={"ID":"b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5","Type":"ContainerStarted","Data":"c1e2983a0b2420d1da7f23366da073f590ed9a06e31db4716916dca5e7960f6e"}
Jul 24 16:30:00.592759 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.592731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v" event={"ID":"80737204-2412-4455-9b04-e4bdf5d0843e","Type":"ContainerStarted","Data":"9041cdeb34cac4cdf23a27c9460241a310d3dfa3c8da241bc84e846128e41060"}
Jul 24 16:30:00.593897 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:00.593878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g" event={"ID":"91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41","Type":"ContainerStarted","Data":"dcbf8798b0dca7881b665214996888c2e2b712aa9d2e1be96581e243f7b2f868"}
Jul 24 16:30:01.601589 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:01.601559    2577 generic.go:356] "Generic (PLEG): container finished" podID="b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5" containerID="f4dbba64e35da777200557a9b2a6903eaea261c39339795e7548de006f962856" exitCode=0
Jul 24 16:30:01.601978 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:01.601647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq" event={"ID":"b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5","Type":"ContainerDied","Data":"f4dbba64e35da777200557a9b2a6903eaea261c39339795e7548de006f962856"}
Jul 24 16:30:01.603162 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:01.603140    2577 generic.go:356] "Generic (PLEG): container finished" podID="80737204-2412-4455-9b04-e4bdf5d0843e" containerID="39acb1395d9709c6b96f496175d2f85985ccdbb1485d20fac6edcf84f8448259" exitCode=0
Jul 24 16:30:01.603239 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:01.603215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v" event={"ID":"80737204-2412-4455-9b04-e4bdf5d0843e","Type":"ContainerDied","Data":"39acb1395d9709c6b96f496175d2f85985ccdbb1485d20fac6edcf84f8448259"}
Jul 24 16:30:01.604507 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:01.604488    2577 generic.go:356] "Generic (PLEG): container finished" podID="91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41" containerID="f4e07c32977ba3cbc83c179db375c6daef3e9bbb0ff4a46fe9268e5ce823db55" exitCode=0
Jul 24 16:30:01.604635 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:01.604510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g" event={"ID":"91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41","Type":"ContainerDied","Data":"f4e07c32977ba3cbc83c179db375c6daef3e9bbb0ff4a46fe9268e5ce823db55"}
Jul 24 16:30:02.757130 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:02.757102    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"
Jul 24 16:30:02.798164 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:02.798145    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"
Jul 24 16:30:02.801114 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:02.801096    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"
Jul 24 16:30:03.617465 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.617434    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"
Jul 24 16:30:03.617666 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.617462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq" event={"ID":"b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5","Type":"ContainerDied","Data":"c1e2983a0b2420d1da7f23366da073f590ed9a06e31db4716916dca5e7960f6e"}
Jul 24 16:30:03.617666 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.617512    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1e2983a0b2420d1da7f23366da073f590ed9a06e31db4716916dca5e7960f6e"
Jul 24 16:30:03.618810 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.618793    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v"
Jul 24 16:30:03.618930 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.618815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748510-r5q9v" event={"ID":"80737204-2412-4455-9b04-e4bdf5d0843e","Type":"ContainerDied","Data":"9041cdeb34cac4cdf23a27c9460241a310d3dfa3c8da241bc84e846128e41060"}
Jul 24 16:30:03.618930 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.618854    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9041cdeb34cac4cdf23a27c9460241a310d3dfa3c8da241bc84e846128e41060"
Jul 24 16:30:03.620065 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.620049    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"
Jul 24 16:30:03.620160 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.620075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g" event={"ID":"91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41","Type":"ContainerDied","Data":"dcbf8798b0dca7881b665214996888c2e2b712aa9d2e1be96581e243f7b2f868"}
Jul 24 16:30:03.620160 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:03.620109    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcbf8798b0dca7881b665214996888c2e2b712aa9d2e1be96581e243f7b2f868"
Jul 24 16:30:07.986210 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:07.986173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:30:30.479993 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:30.479898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:30.495512 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:30.495479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:30:32.640936 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:32.640902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:47.093815 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:30:47.093781    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:31:07.000571 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:31:07.000543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:31:23.011608 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:31:23.011577    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:31:49.185334 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:31:49.185295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:31:58.590309 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:31:58.590246    2577 scope.go:117] "RemoveContainer" containerID="6b6b9586ba58ef6ea922f8751b4e241bc0e2833431eb53260c1fb2a62647c051"
Jul 24 16:31:58.598314 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:31:58.598297    2577 scope.go:117] "RemoveContainer" containerID="f8cfec873385d170e890c6f410171949e4c4f5bf36b62a6e5f67dd9e42ba620e"
Jul 24 16:32:07.785443 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:07.785408    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:32:34.091386 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:34.091353    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:32:37.005050 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:37.005018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:38.002857 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:38.002818    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"]
Jul 24 16:32:38.355517 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:38.355482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:50.692493 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:50.692455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:32:51.885891 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:51.885857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:53.021651 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.021577    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl" podUID="de89db4a-5bee-4477-90b0-201efe550bfd" containerName="main" containerID="cri-o://77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3" gracePeriod=45
Jul 24 16:32:53.277401 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.277349    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:32:53.343704 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-model-cache\") pod \"de89db4a-5bee-4477-90b0-201efe550bfd\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") "
Jul 24 16:32:53.343818 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343727    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr7zn\" (UniqueName: \"kubernetes.io/projected/de89db4a-5bee-4477-90b0-201efe550bfd-kube-api-access-qr7zn\") pod \"de89db4a-5bee-4477-90b0-201efe550bfd\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") "
Jul 24 16:32:53.343818 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343784    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-dshm\") pod \"de89db4a-5bee-4477-90b0-201efe550bfd\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") "
Jul 24 16:32:53.343818 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-home\") pod \"de89db4a-5bee-4477-90b0-201efe550bfd\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") "
Jul 24 16:32:53.343980 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de89db4a-5bee-4477-90b0-201efe550bfd-tls-certs\") pod \"de89db4a-5bee-4477-90b0-201efe550bfd\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") "
Jul 24 16:32:53.343980 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-tmp-dir\") pod \"de89db4a-5bee-4477-90b0-201efe550bfd\" (UID: \"de89db4a-5bee-4477-90b0-201efe550bfd\") "
Jul 24 16:32:53.343980 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.343934    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-model-cache" (OuterVolumeSpecName: "model-cache") pod "de89db4a-5bee-4477-90b0-201efe550bfd" (UID: "de89db4a-5bee-4477-90b0-201efe550bfd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:53.344140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.344047    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-home" (OuterVolumeSpecName: "home") pod "de89db4a-5bee-4477-90b0-201efe550bfd" (UID: "de89db4a-5bee-4477-90b0-201efe550bfd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:53.344202 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.344141    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:32:53.344202 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.344134    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de89db4a-5bee-4477-90b0-201efe550bfd" (UID: "de89db4a-5bee-4477-90b0-201efe550bfd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:53.344202 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.344154    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:32:53.345946 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.345919    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-dshm" (OuterVolumeSpecName: "dshm") pod "de89db4a-5bee-4477-90b0-201efe550bfd" (UID: "de89db4a-5bee-4477-90b0-201efe550bfd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:53.346053 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.345989    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de89db4a-5bee-4477-90b0-201efe550bfd-kube-api-access-qr7zn" (OuterVolumeSpecName: "kube-api-access-qr7zn") pod "de89db4a-5bee-4477-90b0-201efe550bfd" (UID: "de89db4a-5bee-4477-90b0-201efe550bfd"). InnerVolumeSpecName "kube-api-access-qr7zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:53.346053 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.346021    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de89db4a-5bee-4477-90b0-201efe550bfd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de89db4a-5bee-4477-90b0-201efe550bfd" (UID: "de89db4a-5bee-4477-90b0-201efe550bfd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:32:53.444487 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.444464    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de89db4a-5bee-4477-90b0-201efe550bfd-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:32:53.444487 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.444484    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:32:53.444644 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.444495    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr7zn\" (UniqueName: \"kubernetes.io/projected/de89db4a-5bee-4477-90b0-201efe550bfd-kube-api-access-qr7zn\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:32:53.444644 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.444504    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de89db4a-5bee-4477-90b0-201efe550bfd-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:32:53.586557 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.586496    2577 generic.go:356] "Generic (PLEG): container finished" podID="de89db4a-5bee-4477-90b0-201efe550bfd" containerID="77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3" exitCode=0
Jul 24 16:32:53.586557 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.586527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl" event={"ID":"de89db4a-5bee-4477-90b0-201efe550bfd","Type":"ContainerDied","Data":"77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3"}
Jul 24 16:32:53.586557 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.586541    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"
Jul 24 16:32:53.586745 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.586570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl" event={"ID":"de89db4a-5bee-4477-90b0-201efe550bfd","Type":"ContainerDied","Data":"00eb0d3a5ab84b79bb207df229429315a8ae9d5070b423e6479eff42f30661db"}
Jul 24 16:32:53.586745 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.586588    2577 scope.go:117] "RemoveContainer" containerID="77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3"
Jul 24 16:32:53.595789 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.595771    2577 scope.go:117] "RemoveContainer" containerID="77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3"
Jul 24 16:32:53.596039 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:32:53.596022    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3\": container with ID starting with 77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3 not found: ID does not exist" containerID="77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3"
Jul 24 16:32:53.596097 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.596046    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3"} err="failed to get container status \"77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3\": rpc error: code = NotFound desc = could not find container \"77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3\": container with ID starting with 77923288204d96976064ade0959433ae127ae77bfeeecda8026cc4d8924e96d3 not found: ID does not exist"
Jul 24 16:32:53.606955 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.606936    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"]
Jul 24 16:32:53.610828 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:53.610808    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/auth-test-model-675edf-kserve-6f844d989f-p8btl"]
Jul 24 16:32:54.238258 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:54.238219    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de89db4a-5bee-4477-90b0-201efe550bfd" path="/var/lib/kubelet/pods/de89db4a-5bee-4477-90b0-201efe550bfd/volumes"
Jul 24 16:32:58.626746 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:58.626657    2577 scope.go:117] "RemoveContainer" containerID="285513c69d4502d4c19e146b38cbf8df79d471aa8b345933006192a775f69cc3"
Jul 24 16:32:58.639333 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:32:58.634716    2577 scope.go:117] "RemoveContainer" containerID="939e5460b3f9096c0c5b43b091d5224b5480876a33f1334b59cb3face49d8508"
Jul 24 16:33:02.992174 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:02.992140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:33:39.301528 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.301489    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"]
Jul 24 16:33:39.302109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302046    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5" containerName="cleanup"
Jul 24 16:33:39.302109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302061    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5" containerName="cleanup"
Jul 24 16:33:39.302109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302078    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80737204-2412-4455-9b04-e4bdf5d0843e" containerName="cleanup"
Jul 24 16:33:39.302109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302086    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="80737204-2412-4455-9b04-e4bdf5d0843e" containerName="cleanup"
Jul 24 16:33:39.302109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302102    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41" containerName="cleanup"
Jul 24 16:33:39.302109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302110    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41" containerName="cleanup"
Jul 24 16:33:39.302298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302128    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de89db4a-5bee-4477-90b0-201efe550bfd" containerName="main"
Jul 24 16:33:39.302298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302137    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="de89db4a-5bee-4477-90b0-201efe550bfd" containerName="main"
Jul 24 16:33:39.302298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302216    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="de89db4a-5bee-4477-90b0-201efe550bfd" containerName="main"
Jul 24 16:33:39.302298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302227    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41" containerName="cleanup"
Jul 24 16:33:39.302298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302236    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5" containerName="cleanup"
Jul 24 16:33:39.302298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.302249    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="80737204-2412-4455-9b04-e4bdf5d0843e" containerName="cleanup"
Jul 24 16:33:39.305472 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.305454    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.308094 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.308068    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"openshift-service-ca.crt\""
Jul 24 16:33:39.308227 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.308132    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"sub-test-model-675edf-kserve-self-signed-certs\""
Jul 24 16:33:39.308227 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.308075    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"default-dockercfg-rb79d\""
Jul 24 16:33:39.309342 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.309326    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"kube-root-ca.crt\""
Jul 24 16:33:39.312338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.312314    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"]
Jul 24 16:33:39.327572 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.327550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:33:39.367833 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.367803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tls-certs\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.367921 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.367845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-home\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.367921 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.367877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-model-cache\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.367921 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.367898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tmp-dir\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.368031 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.367955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlkwj\" (UniqueName: \"kubernetes.io/projected/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-kube-api-access-jlkwj\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.368031 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.368002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-dshm\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.468820 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.468796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlkwj\" (UniqueName: \"kubernetes.io/projected/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-kube-api-access-jlkwj\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.468925 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.468840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-dshm\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.468925 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.468865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tls-certs\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.468925 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.468883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-home\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.468925 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.468921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-model-cache\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.469117 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.468935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tmp-dir\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.469303 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.469279    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tmp-dir\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.469420 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.469314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-model-cache\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.469420 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.469356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-home\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.471037 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.471019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-dshm\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.471370 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.471351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tls-certs\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.476289 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.476266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlkwj\" (UniqueName: \"kubernetes.io/projected/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-kube-api-access-jlkwj\") pod \"sub-test-model-675edf-kserve-5446986d5-flpgb\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") " pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.617067 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.617002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:39.747382 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.747359    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"]
Jul 24 16:33:39.749229 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:33:39.749203    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode92d6103_e910_4ebc_9cd9_5fe2d0965dbf.slice/crio-8d8067f08e5591db77d9d0fb7f48ed89b0153efbf2640789539e7fab0575f851 WatchSource:0}: Error finding container 8d8067f08e5591db77d9d0fb7f48ed89b0153efbf2640789539e7fab0575f851: Status 404 returned error can't find the container with id 8d8067f08e5591db77d9d0fb7f48ed89b0153efbf2640789539e7fab0575f851
Jul 24 16:33:39.835890 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.835862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb" event={"ID":"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf","Type":"ContainerStarted","Data":"9dda6b5ce5231898729cee94fdf917d21b92c22b65b2b6021260af988d493a39"}
Jul 24 16:33:39.836009 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.835902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb" event={"ID":"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf","Type":"ContainerStarted","Data":"8d8067f08e5591db77d9d0fb7f48ed89b0153efbf2640789539e7fab0575f851"}
Jul 24 16:33:39.853459 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:39.853417    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb" podStartSLOduration=0.853401326 podStartE2EDuration="853.401326ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:33:39.851772734 +0000 UTC m=+2802.179337275" watchObservedRunningTime="2026-07-24 16:33:39.853401326 +0000 UTC m=+2802.180965864"
Jul 24 16:33:43.398131 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:43.398094    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:33:49.617755 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:49.617720    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:49.618164 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:49.617766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:49.630276 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:49.630255    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:33:49.900654 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:33:49.900570    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:34:16.703402 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:16.703368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:34:36.185299 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:36.185265    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:34:42.229534 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:42.229496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:34:43.074388 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:43.074359    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"]
Jul 24 16:34:43.102787 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:43.102756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:34:43.447580 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:43.447507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:34:43.547766 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:43.547737    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:34:53.910434 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:53.910357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:34:58.092802 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.092756    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb" podUID="e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" containerName="main" containerID="cri-o://9dda6b5ce5231898729cee94fdf917d21b92c22b65b2b6021260af988d493a39" gracePeriod=45
Jul 24 16:34:58.252072 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.252043    2577 generic.go:356] "Generic (PLEG): container finished" podID="e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" containerID="9dda6b5ce5231898729cee94fdf917d21b92c22b65b2b6021260af988d493a39" exitCode=0
Jul 24 16:34:58.252230 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.252119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb" event={"ID":"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf","Type":"ContainerDied","Data":"9dda6b5ce5231898729cee94fdf917d21b92c22b65b2b6021260af988d493a39"}
Jul 24 16:34:58.340227 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.340200    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:34:58.389863 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.389799    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tmp-dir\") pod \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") "
Jul 24 16:34:58.389985 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.389876    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlkwj\" (UniqueName: \"kubernetes.io/projected/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-kube-api-access-jlkwj\") pod \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") "
Jul 24 16:34:58.389985 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.389906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-home\") pod \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") "
Jul 24 16:34:58.389985 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.389936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-dshm\") pod \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") "
Jul 24 16:34:58.389985 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.389966    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tls-certs\") pod \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") "
Jul 24 16:34:58.390192 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.390014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-model-cache\") pod \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\" (UID: \"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf\") "
Jul 24 16:34:58.390192 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.390056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" (UID: "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:34:58.390192 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.390128    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-home" (OuterVolumeSpecName: "home") pod "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" (UID: "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:34:58.390353 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.390277    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:34:58.390353 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.390281    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-model-cache" (OuterVolumeSpecName: "model-cache") pod "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" (UID: "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:34:58.390353 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.390295    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:34:58.391960 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.391936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" (UID: "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:34:58.392059 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.391987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-kube-api-access-jlkwj" (OuterVolumeSpecName: "kube-api-access-jlkwj") pod "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" (UID: "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf"). InnerVolumeSpecName "kube-api-access-jlkwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:34:58.392059 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.392022    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-dshm" (OuterVolumeSpecName: "dshm") pod "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" (UID: "e92d6103-e910-4ebc-9cd9-5fe2d0965dbf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:34:58.491365 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.491338    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlkwj\" (UniqueName: \"kubernetes.io/projected/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-kube-api-access-jlkwj\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:34:58.491365 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.491359    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:34:58.491365 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.491369    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:34:58.491561 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:58.491379    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:34:59.258999 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:59.258964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb" event={"ID":"e92d6103-e910-4ebc-9cd9-5fe2d0965dbf","Type":"ContainerDied","Data":"8d8067f08e5591db77d9d0fb7f48ed89b0153efbf2640789539e7fab0575f851"}
Jul 24 16:34:59.259434 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:59.259002    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"
Jul 24 16:34:59.259434 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:59.259015    2577 scope.go:117] "RemoveContainer" containerID="9dda6b5ce5231898729cee94fdf917d21b92c22b65b2b6021260af988d493a39"
Jul 24 16:34:59.279043 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:59.279021    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"]
Jul 24 16:34:59.288112 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:34:59.288091    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/sub-test-model-675edf-kserve-5446986d5-flpgb"]
Jul 24 16:35:00.238331 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:00.238298    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" path="/var/lib/kubelet/pods/e92d6103-e910-4ebc-9cd9-5fe2d0965dbf/volumes"
Jul 24 16:35:04.532926 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:04.532894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:35:07.994713 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:07.994672    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:35:22.302513 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.302484    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"]
Jul 24 16:35:22.302923 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.302882    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" containerName="main"
Jul 24 16:35:22.302923 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.302896    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" containerName="main"
Jul 24 16:35:22.302996 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.302959    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92d6103-e910-4ebc-9cd9-5fe2d0965dbf" containerName="main"
Jul 24 16:35:22.305914 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.305900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.308439 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.308414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"rate-test-model-675edf-kserve-self-signed-certs\""
Jul 24 16:35:22.308582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.308416    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"default-dockercfg-rb79d\""
Jul 24 16:35:22.308582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.308422    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"kube-root-ca.crt\""
Jul 24 16:35:22.308582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.308424    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-675edf\"/\"openshift-service-ca.crt\""
Jul 24 16:35:22.316799 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.316777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"]
Jul 24 16:35:22.332702 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.332676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:35:22.480680 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.480655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-tmp-dir\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.480808 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.480692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0295a7e6-3571-49a5-a129-b4dffba271b2-tls-certs\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.480808 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.480720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl42k\" (UniqueName: \"kubernetes.io/projected/0295a7e6-3571-49a5-a129-b4dffba271b2-kube-api-access-kl42k\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.480946 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.480805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-home\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.480946 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.480838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-model-cache\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.480946 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.480925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-dshm\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.581850 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.581794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-dshm\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.581850 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.581833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-tmp-dir\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582036 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.581860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0295a7e6-3571-49a5-a129-b4dffba271b2-tls-certs\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582036 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.581890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl42k\" (UniqueName: \"kubernetes.io/projected/0295a7e6-3571-49a5-a129-b4dffba271b2-kube-api-access-kl42k\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582036 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.581907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-home\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582036 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.581921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-model-cache\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582314 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.582284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-tmp-dir\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582429 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.582344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-home\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.582429 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.582375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-model-cache\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.583922 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.583905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-dshm\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.584294 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.584276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0295a7e6-3571-49a5-a129-b4dffba271b2-tls-certs\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.590716 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.590695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl42k\" (UniqueName: \"kubernetes.io/projected/0295a7e6-3571-49a5-a129-b4dffba271b2-kube-api-access-kl42k\") pod \"rate-test-model-675edf-kserve-75947dd787-548nt\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") " pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.615690 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.615669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:22.749415 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:22.749389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"]
Jul 24 16:35:22.750452 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:35:22.750412    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0295a7e6_3571_49a5_a129_b4dffba271b2.slice/crio-b098d0d66213b93439ebd51098c0b306ee55de42eb59f3d7f36b7260df122c4f WatchSource:0}: Error finding container b098d0d66213b93439ebd51098c0b306ee55de42eb59f3d7f36b7260df122c4f: Status 404 returned error can't find the container with id b098d0d66213b93439ebd51098c0b306ee55de42eb59f3d7f36b7260df122c4f
Jul 24 16:35:23.393020 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:23.392987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt" event={"ID":"0295a7e6-3571-49a5-a129-b4dffba271b2","Type":"ContainerStarted","Data":"af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95"}
Jul 24 16:35:23.393413 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:23.393032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt" event={"ID":"0295a7e6-3571-49a5-a129-b4dffba271b2","Type":"ContainerStarted","Data":"b098d0d66213b93439ebd51098c0b306ee55de42eb59f3d7f36b7260df122c4f"}
Jul 24 16:35:23.410541 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:23.410491    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt" podStartSLOduration=1.410476138 podStartE2EDuration="1.410476138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:35:23.409355412 +0000 UTC m=+2905.736919965" watchObservedRunningTime="2026-07-24 16:35:23.410476138 +0000 UTC m=+2905.738040677"
Jul 24 16:35:26.087939 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:26.087911    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:35:32.616838 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:32.616808    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:32.617233 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:32.616852    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:32.629130 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:32.629104    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:35:33.460911 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:35:33.460878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:36:01.988854 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:01.988821    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:36:17.201045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:17.201012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:36:33.991570 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:33.991536    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:36:37.738517 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:37.738465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:36:38.602663 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:38.602630    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"]
Jul 24 16:36:38.621159 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:38.621130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:36:50.994996 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:50.994952    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:36:53.621714 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.621677    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt" podUID="0295a7e6-3571-49a5-a129-b4dffba271b2" containerName="main" containerID="cri-o://af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95" gracePeriod=45
Jul 24 16:36:53.867304 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.867281    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:36:53.889327 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.889271    2577 generic.go:356] "Generic (PLEG): container finished" podID="0295a7e6-3571-49a5-a129-b4dffba271b2" containerID="af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95" exitCode=0
Jul 24 16:36:53.889327 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.889306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"
Jul 24 16:36:53.889481 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.889345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt" event={"ID":"0295a7e6-3571-49a5-a129-b4dffba271b2","Type":"ContainerDied","Data":"af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95"}
Jul 24 16:36:53.889481 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.889381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt" event={"ID":"0295a7e6-3571-49a5-a129-b4dffba271b2","Type":"ContainerDied","Data":"b098d0d66213b93439ebd51098c0b306ee55de42eb59f3d7f36b7260df122c4f"}
Jul 24 16:36:53.889481 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.889397    2577 scope.go:117] "RemoveContainer" containerID="af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95"
Jul 24 16:36:53.898296 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.898277    2577 scope.go:117] "RemoveContainer" containerID="af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95"
Jul 24 16:36:53.898566 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:36:53.898546    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95\": container with ID starting with af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95 not found: ID does not exist" containerID="af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95"
Jul 24 16:36:53.898679 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.898572    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95"} err="failed to get container status \"af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95\": rpc error: code = NotFound desc = could not find container \"af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95\": container with ID starting with af0a847c562977bbee8b0fb57114fde475d4c5337314e40520b8501980bfdb95 not found: ID does not exist"
Jul 24 16:36:53.966965 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.966947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-home\") pod \"0295a7e6-3571-49a5-a129-b4dffba271b2\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") "
Jul 24 16:36:53.967047 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.966974    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl42k\" (UniqueName: \"kubernetes.io/projected/0295a7e6-3571-49a5-a129-b4dffba271b2-kube-api-access-kl42k\") pod \"0295a7e6-3571-49a5-a129-b4dffba271b2\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") "
Jul 24 16:36:53.967047 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-tmp-dir\") pod \"0295a7e6-3571-49a5-a129-b4dffba271b2\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") "
Jul 24 16:36:53.967047 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967044    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0295a7e6-3571-49a5-a129-b4dffba271b2-tls-certs\") pod \"0295a7e6-3571-49a5-a129-b4dffba271b2\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") "
Jul 24 16:36:53.967173 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-dshm\") pod \"0295a7e6-3571-49a5-a129-b4dffba271b2\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") "
Jul 24 16:36:53.967173 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967127    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-model-cache\") pod \"0295a7e6-3571-49a5-a129-b4dffba271b2\" (UID: \"0295a7e6-3571-49a5-a129-b4dffba271b2\") "
Jul 24 16:36:53.967282 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967199    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-home" (OuterVolumeSpecName: "home") pod "0295a7e6-3571-49a5-a129-b4dffba271b2" (UID: "0295a7e6-3571-49a5-a129-b4dffba271b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:53.967339 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967292    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0295a7e6-3571-49a5-a129-b4dffba271b2" (UID: "0295a7e6-3571-49a5-a129-b4dffba271b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:53.967440 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967420    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:36:53.967502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967445    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:36:53.967502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.967419    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "0295a7e6-3571-49a5-a129-b4dffba271b2" (UID: "0295a7e6-3571-49a5-a129-b4dffba271b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:53.969022 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.968999    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0295a7e6-3571-49a5-a129-b4dffba271b2-kube-api-access-kl42k" (OuterVolumeSpecName: "kube-api-access-kl42k") pod "0295a7e6-3571-49a5-a129-b4dffba271b2" (UID: "0295a7e6-3571-49a5-a129-b4dffba271b2"). InnerVolumeSpecName "kube-api-access-kl42k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:53.969137 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.969119    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-dshm" (OuterVolumeSpecName: "dshm") pod "0295a7e6-3571-49a5-a129-b4dffba271b2" (UID: "0295a7e6-3571-49a5-a129-b4dffba271b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:53.969192 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:53.969141    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0295a7e6-3571-49a5-a129-b4dffba271b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0295a7e6-3571-49a5-a129-b4dffba271b2" (UID: "0295a7e6-3571-49a5-a129-b4dffba271b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:36:54.068582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.068558    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0295a7e6-3571-49a5-a129-b4dffba271b2-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:36:54.068582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.068578    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:36:54.068718 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.068586    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0295a7e6-3571-49a5-a129-b4dffba271b2-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:36:54.068718 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.068594    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl42k\" (UniqueName: \"kubernetes.io/projected/0295a7e6-3571-49a5-a129-b4dffba271b2-kube-api-access-kl42k\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:36:54.210143 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.210120    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"]
Jul 24 16:36:54.215073 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.215053    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-675edf/rate-test-model-675edf-kserve-75947dd787-548nt"]
Jul 24 16:36:54.238897 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:54.238873    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0295a7e6-3571-49a5-a129-b4dffba271b2" path="/var/lib/kubelet/pods/0295a7e6-3571-49a5-a129-b4dffba271b2/volumes"
Jul 24 16:36:57.762448 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:57.762413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:36:58.058717 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:58.058685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:36:58.936888 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:36:58.936850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:00.989857 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:00.989804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:01.130753 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.130723    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:01.218527 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.218492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:01.319862 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.319836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:01.716004 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.715927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:01.814449 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.814416    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"]
Jul 24 16:37:01.814807 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.814783    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0295a7e6-3571-49a5-a129-b4dffba271b2" containerName="main"
Jul 24 16:37:01.814807 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.814801    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0295a7e6-3571-49a5-a129-b4dffba271b2" containerName="main"
Jul 24 16:37:01.814971 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.814868    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0295a7e6-3571-49a5-a129-b4dffba271b2" containerName="main"
Jul 24 16:37:01.816711 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.816660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:01.819392 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.819375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:01.821311 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.821290    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-ae4522\""
Jul 24 16:37:01.821412 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.821293    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-ae4522-dockercfg-sshnb\""
Jul 24 16:37:01.830626 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.830597    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"]
Jul 24 16:37:01.841585 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.841555    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"]
Jul 24 16:37:01.845011 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.844993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:01.862460 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.862437    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"]
Jul 24 16:37:01.932292 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.932261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6b621d9-b470-4a23-8792-1b09efe60ae7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:01.932418 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.932311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/834cb701-e4c4-4f16-aafa-67f48494b7b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:01.932418 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.932336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zxx\" (UniqueName: \"kubernetes.io/projected/834cb701-e4c4-4f16-aafa-67f48494b7b8-kube-api-access-m8zxx\") pod \"payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:01.932418 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:01.932366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmlr\" (UniqueName: \"kubernetes.io/projected/d6b621d9-b470-4a23-8792-1b09efe60ae7-kube-api-access-tnmlr\") pod \"payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.033493 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.033463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6b621d9-b470-4a23-8792-1b09efe60ae7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.033928 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.033520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/834cb701-e4c4-4f16-aafa-67f48494b7b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:02.033928 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.033547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zxx\" (UniqueName: \"kubernetes.io/projected/834cb701-e4c4-4f16-aafa-67f48494b7b8-kube-api-access-m8zxx\") pod \"payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:02.033928 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.033638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmlr\" (UniqueName: \"kubernetes.io/projected/d6b621d9-b470-4a23-8792-1b09efe60ae7-kube-api-access-tnmlr\") pod \"payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.034275 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.034259    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6b621d9-b470-4a23-8792-1b09efe60ae7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.034324 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.034272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/834cb701-e4c4-4f16-aafa-67f48494b7b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:02.041961 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.041945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmlr\" (UniqueName: \"kubernetes.io/projected/d6b621d9-b470-4a23-8792-1b09efe60ae7-kube-api-access-tnmlr\") pod \"payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.042120 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.042100    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zxx\" (UniqueName: \"kubernetes.io/projected/834cb701-e4c4-4f16-aafa-67f48494b7b8-kube-api-access-m8zxx\") pod \"payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:02.130810 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.130786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:02.155096 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.155074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.277400 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.277371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"]
Jul 24 16:37:02.277991 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:37:02.277968    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834cb701_e4c4_4f16_aafa_67f48494b7b8.slice/crio-cbe6f91181e914074638da4fcdccdadbfb33a2882a51d778cb2467da24420bf8 WatchSource:0}: Error finding container cbe6f91181e914074638da4fcdccdadbfb33a2882a51d778cb2467da24420bf8: Status 404 returned error can't find the container with id cbe6f91181e914074638da4fcdccdadbfb33a2882a51d778cb2467da24420bf8
Jul 24 16:37:02.303345 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.303322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"]
Jul 24 16:37:02.320820 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:37:02.320799    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6b621d9_b470_4a23_8792_1b09efe60ae7.slice/crio-9df215569667bfbcc753a42c034c93ffa635573fc77e95684f32b61f2039e2da WatchSource:0}: Error finding container 9df215569667bfbcc753a42c034c93ffa635573fc77e95684f32b61f2039e2da: Status 404 returned error can't find the container with id 9df215569667bfbcc753a42c034c93ffa635573fc77e95684f32b61f2039e2da
Jul 24 16:37:02.952243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.952212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq" event={"ID":"834cb701-e4c4-4f16-aafa-67f48494b7b8","Type":"ContainerStarted","Data":"d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54"}
Jul 24 16:37:02.952424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.952259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq" event={"ID":"834cb701-e4c4-4f16-aafa-67f48494b7b8","Type":"ContainerStarted","Data":"cbe6f91181e914074638da4fcdccdadbfb33a2882a51d778cb2467da24420bf8"}
Jul 24 16:37:02.952424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.952389    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:02.953796 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.953765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4" event={"ID":"d6b621d9-b470-4a23-8792-1b09efe60ae7","Type":"ContainerStarted","Data":"76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad"}
Jul 24 16:37:02.953901 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.953816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4" event={"ID":"d6b621d9-b470-4a23-8792-1b09efe60ae7","Type":"ContainerStarted","Data":"9df215569667bfbcc753a42c034c93ffa635573fc77e95684f32b61f2039e2da"}
Jul 24 16:37:02.953901 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.953843    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:02.968773 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.968728    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq" podStartSLOduration=1.968715455 podStartE2EDuration="1.968715455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:02.967023387 +0000 UTC m=+3005.294587929" watchObservedRunningTime="2026-07-24 16:37:02.968715455 +0000 UTC m=+3005.296279992"
Jul 24 16:37:02.986185 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:02.986150    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4" podStartSLOduration=1.9861422119999999 podStartE2EDuration="1.986142212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:02.984754245 +0000 UTC m=+3005.312318810" watchObservedRunningTime="2026-07-24 16:37:02.986142212 +0000 UTC m=+3005.313706795"
Jul 24 16:37:05.442206 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:05.442179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:09.682635 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:09.682578    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:37:13.967324 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:13.967293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:37:13.967830 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:13.967372    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:37:16.318146 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:16.318105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:16.622562 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:16.622473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:17.047694 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:17.047662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:18.256693 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:18.256658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:20.127629 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:20.127572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:20.235471 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:20.235424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:20.725726 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:20.725682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:21.194804 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.194772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:21.286903 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.286871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:21.327300 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.327273    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"]
Jul 24 16:37:21.330670 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.330656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.332628 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.332597    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d71769\""
Jul 24 16:37:21.332751 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.332710    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d71769-dockercfg-nrktd\""
Jul 24 16:37:21.342678 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.342655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"]
Jul 24 16:37:21.353607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.353581    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"]
Jul 24 16:37:21.356823 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.356809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.365417 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.365400    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"]
Jul 24 16:37:21.389110 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.389078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b628a59-f0fa-4b74-beeb-9960d40f2c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.389225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.389122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1138f6b-274d-4ca0-be59-b398b13a7184-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.389225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.389156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh2x9\" (UniqueName: \"kubernetes.io/projected/c1138f6b-274d-4ca0-be59-b398b13a7184-kube-api-access-dh2x9\") pod \"payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.389225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.389192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7klb\" (UniqueName: \"kubernetes.io/projected/8b628a59-f0fa-4b74-beeb-9960d40f2c49-kube-api-access-k7klb\") pod \"payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.490135 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.490066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b628a59-f0fa-4b74-beeb-9960d40f2c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.490135 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.490103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1138f6b-274d-4ca0-be59-b398b13a7184-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.490135 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.490132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh2x9\" (UniqueName: \"kubernetes.io/projected/c1138f6b-274d-4ca0-be59-b398b13a7184-kube-api-access-dh2x9\") pod \"payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.490332 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.490173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7klb\" (UniqueName: \"kubernetes.io/projected/8b628a59-f0fa-4b74-beeb-9960d40f2c49-kube-api-access-k7klb\") pod \"payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.490778 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.490761    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b628a59-f0fa-4b74-beeb-9960d40f2c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.490884 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.490867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1138f6b-274d-4ca0-be59-b398b13a7184-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.497391 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.497371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7klb\" (UniqueName: \"kubernetes.io/projected/8b628a59-f0fa-4b74-beeb-9960d40f2c49-kube-api-access-k7klb\") pod \"payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.497503 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.497486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh2x9\" (UniqueName: \"kubernetes.io/projected/c1138f6b-274d-4ca0-be59-b398b13a7184-kube-api-access-dh2x9\") pod \"payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.565193 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.565167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:21.643088 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.643067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:21.668695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.668671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:21.988434 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:21.988409    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"]
Jul 24 16:37:21.990009 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:37:21.989980    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1138f6b_274d_4ca0_be59_b398b13a7184.slice/crio-389c54f9c88d4bed8bc81b3c8303c64e960ef90f4aa2124b3a245f4124a55699 WatchSource:0}: Error finding container 389c54f9c88d4bed8bc81b3c8303c64e960ef90f4aa2124b3a245f4124a55699: Status 404 returned error can't find the container with id 389c54f9c88d4bed8bc81b3c8303c64e960ef90f4aa2124b3a245f4124a55699
Jul 24 16:37:22.014784 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:22.014761    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"]
Jul 24 16:37:22.025599 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:37:22.025576    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b628a59_f0fa_4b74_beeb_9960d40f2c49.slice/crio-2cc2c5ad7142a75119712a12900139ad3a7a0ba0b7a5548a95b429ed1d001b9c WatchSource:0}: Error finding container 2cc2c5ad7142a75119712a12900139ad3a7a0ba0b7a5548a95b429ed1d001b9c: Status 404 returned error can't find the container with id 2cc2c5ad7142a75119712a12900139ad3a7a0ba0b7a5548a95b429ed1d001b9c
Jul 24 16:37:22.069043 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:22.069017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz" event={"ID":"8b628a59-f0fa-4b74-beeb-9960d40f2c49","Type":"ContainerStarted","Data":"2cc2c5ad7142a75119712a12900139ad3a7a0ba0b7a5548a95b429ed1d001b9c"}
Jul 24 16:37:22.070783 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:22.070756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv" event={"ID":"c1138f6b-274d-4ca0-be59-b398b13a7184","Type":"ContainerStarted","Data":"c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8"}
Jul 24 16:37:22.070879 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:22.070807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv" event={"ID":"c1138f6b-274d-4ca0-be59-b398b13a7184","Type":"ContainerStarted","Data":"389c54f9c88d4bed8bc81b3c8303c64e960ef90f4aa2124b3a245f4124a55699"}
Jul 24 16:37:22.070948 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:22.070897    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:22.086669 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:22.086626    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv" podStartSLOduration=1.086598263 podStartE2EDuration="1.086598263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:22.084810893 +0000 UTC m=+3024.412375433" watchObservedRunningTime="2026-07-24 16:37:22.086598263 +0000 UTC m=+3024.414162800"
Jul 24 16:37:23.077892 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:23.077853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz" event={"ID":"8b628a59-f0fa-4b74-beeb-9960d40f2c49","Type":"ContainerStarted","Data":"1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277"}
Jul 24 16:37:23.078329 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:23.077970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:23.093034 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:23.092983    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz" podStartSLOduration=2.092967726 podStartE2EDuration="2.092967726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:23.091931398 +0000 UTC m=+3025.419495957" watchObservedRunningTime="2026-07-24 16:37:23.092967726 +0000 UTC m=+3025.420532269"
Jul 24 16:37:23.895609 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:23.895584    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:37:24.648064 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:24.648031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:33.080125 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:33.080095    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:37:34.085289 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:34.085257    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:37:41.830964 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:41.830934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv_c1138f6b-274d-4ca0-be59-b398b13a7184/payload-processing/0.log"
Jul 24 16:37:45.588917 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.588882    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"]
Jul 24 16:37:45.592225 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.592209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.593964 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.593943    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ae4522\"/\"ipp-route-ae4522-kserve-self-signed-certs\""
Jul 24 16:37:45.594468 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.594447    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ae4522\"/\"kube-root-ca.crt\""
Jul 24 16:37:45.594584 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.594521    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ae4522\"/\"openshift-service-ca.crt\""
Jul 24 16:37:45.594743 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.594727    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ae4522\"/\"default-dockercfg-qgx4b\""
Jul 24 16:37:45.600061 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.600039    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"]
Jul 24 16:37:45.624522 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.624492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:37:45.686298 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.686276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tmp-dir\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.686397 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.686300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-model-cache\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.686397 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.686329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-dshm\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.686397 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.686376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-home\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.686536 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.686426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tls-certs\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.686536 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.686449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djwhl\" (UniqueName: \"kubernetes.io/projected/b5f285ab-a6b2-4f4d-8788-5b24656b2884-kube-api-access-djwhl\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.786947 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.786918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-model-cache\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787058 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.786956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-dshm\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787058 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.786975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-home\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787058 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.787015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tls-certs\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787058 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.787039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djwhl\" (UniqueName: \"kubernetes.io/projected/b5f285ab-a6b2-4f4d-8788-5b24656b2884-kube-api-access-djwhl\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787204 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.787123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tmp-dir\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787342 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.787319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-model-cache\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787473 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.787454    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-home\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.787473 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.787407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tmp-dir\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.789319 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.789299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-dshm\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.789520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.789501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tls-certs\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.793797 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.793781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djwhl\" (UniqueName: \"kubernetes.io/projected/b5f285ab-a6b2-4f4d-8788-5b24656b2884-kube-api-access-djwhl\") pod \"ipp-route-ae4522-kserve-5bc98f59c9-n62f4\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") " pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:45.904338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:45.904282    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:46.037068 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:46.037035    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"]
Jul 24 16:37:46.038533 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:37:46.038510    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5f285ab_a6b2_4f4d_8788_5b24656b2884.slice/crio-d03c2d5704e2e3df2dffc1260c7837d891e71cadaaa29cf9a5cc1a40783fc18d WatchSource:0}: Error finding container d03c2d5704e2e3df2dffc1260c7837d891e71cadaaa29cf9a5cc1a40783fc18d: Status 404 returned error can't find the container with id d03c2d5704e2e3df2dffc1260c7837d891e71cadaaa29cf9a5cc1a40783fc18d
Jul 24 16:37:46.207124 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:46.207037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4" event={"ID":"b5f285ab-a6b2-4f4d-8788-5b24656b2884","Type":"ContainerStarted","Data":"0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe"}
Jul 24 16:37:46.207124 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:46.207080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4" event={"ID":"b5f285ab-a6b2-4f4d-8788-5b24656b2884","Type":"ContainerStarted","Data":"d03c2d5704e2e3df2dffc1260c7837d891e71cadaaa29cf9a5cc1a40783fc18d"}
Jul 24 16:37:46.223184 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:46.223134    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4" podStartSLOduration=1.223119667 podStartE2EDuration="1.223119667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:46.220327007 +0000 UTC m=+3048.547891548" watchObservedRunningTime="2026-07-24 16:37:46.223119667 +0000 UTC m=+3048.550684205"
Jul 24 16:37:49.588174 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:49.588143    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:37:55.904559 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:55.904527    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:55.904988 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:55.904574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:55.916590 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:55.916566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:37:56.273356 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:37:56.273334    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:38:01.052999 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:01.052958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:11.710140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:11.710105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:11.813072 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:11.813048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:12.413444 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:12.413411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:12.513769 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:12.513741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:17.791474 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:17.791439    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:38:28.703741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:28.703710    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4_d6b621d9-b470-4a23-8792-1b09efe60ae7/payload-processing/0.log"
Jul 24 16:38:28.933984 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:28.933961    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv_c1138f6b-274d-4ca0-be59-b398b13a7184/payload-processing/0.log"
Jul 24 16:38:29.191754 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:29.191725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:29.496160 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:29.496080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:29.813757 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:29.813726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:32.598063 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:32.598030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:32.696975 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:32.696946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:34.935484 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:34.935453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:35.171874 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.171838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:35.481077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.481045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:35.575891 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.575862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:35.576459 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.576438    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"]
Jul 24 16:38:35.580460 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.580440    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.582274 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.582253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-292d47\""
Jul 24 16:38:35.582394 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.582255    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-292d47-dockercfg-4fkb2\""
Jul 24 16:38:35.589670 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.589649    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"]
Jul 24 16:38:35.624784 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.624756    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"]
Jul 24 16:38:35.628060 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.628046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.632727 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.632708    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"]
Jul 24 16:38:35.671264 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.671242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mblnr\" (UniqueName: \"kubernetes.io/projected/20749917-6490-4cb3-a8da-6a864f4a91ac-kube-api-access-mblnr\") pod \"payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.671384 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.671272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354115bd-b27c-44cd-a27c-0c4d34692b1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.671384 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.671305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20749917-6490-4cb3-a8da-6a864f4a91ac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.671479 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.671405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2dv9\" (UniqueName: \"kubernetes.io/projected/354115bd-b27c-44cd-a27c-0c4d34692b1c-kube-api-access-b2dv9\") pod \"payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.772187 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.772162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mblnr\" (UniqueName: \"kubernetes.io/projected/20749917-6490-4cb3-a8da-6a864f4a91ac-kube-api-access-mblnr\") pod \"payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.772338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.772194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354115bd-b27c-44cd-a27c-0c4d34692b1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.772338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.772230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20749917-6490-4cb3-a8da-6a864f4a91ac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.772338 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.772286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2dv9\" (UniqueName: \"kubernetes.io/projected/354115bd-b27c-44cd-a27c-0c4d34692b1c-kube-api-access-b2dv9\") pod \"payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.773183 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.773161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354115bd-b27c-44cd-a27c-0c4d34692b1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.773242 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.773161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20749917-6490-4cb3-a8da-6a864f4a91ac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.780160 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.780133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2dv9\" (UniqueName: \"kubernetes.io/projected/354115bd-b27c-44cd-a27c-0c4d34692b1c-kube-api-access-b2dv9\") pod \"payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.780258 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.780170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mblnr\" (UniqueName: \"kubernetes.io/projected/20749917-6490-4cb3-a8da-6a864f4a91ac-kube-api-access-mblnr\") pod \"payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:35.893034 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.893013    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:35.941228 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:35.941203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:36.039967 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.039937    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"]
Jul 24 16:38:36.041860 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:38:36.041813    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354115bd_b27c_44cd_a27c_0c4d34692b1c.slice/crio-94ea6d7b5dbfab499638d1328f4d9ba8b6fa89a2dfbf5707dff382c6c541fbf9 WatchSource:0}: Error finding container 94ea6d7b5dbfab499638d1328f4d9ba8b6fa89a2dfbf5707dff382c6c541fbf9: Status 404 returned error can't find the container with id 94ea6d7b5dbfab499638d1328f4d9ba8b6fa89a2dfbf5707dff382c6c541fbf9
Jul 24 16:38:36.295910 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.295888    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"]
Jul 24 16:38:36.297466 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:38:36.297432    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20749917_6490_4cb3_a8da_6a864f4a91ac.slice/crio-3d7417f46699caceea7f21bd2b00e413719924fe017c9ab1e9dc223ffd1d5c96 WatchSource:0}: Error finding container 3d7417f46699caceea7f21bd2b00e413719924fe017c9ab1e9dc223ffd1d5c96: Status 404 returned error can't find the container with id 3d7417f46699caceea7f21bd2b00e413719924fe017c9ab1e9dc223ffd1d5c96
Jul 24 16:38:36.493283 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.493249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8" event={"ID":"20749917-6490-4cb3-a8da-6a864f4a91ac","Type":"ContainerStarted","Data":"e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d"}
Jul 24 16:38:36.493456 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.493309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8" event={"ID":"20749917-6490-4cb3-a8da-6a864f4a91ac","Type":"ContainerStarted","Data":"3d7417f46699caceea7f21bd2b00e413719924fe017c9ab1e9dc223ffd1d5c96"}
Jul 24 16:38:36.493456 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.493431    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:36.494863 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.494841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6" event={"ID":"354115bd-b27c-44cd-a27c-0c4d34692b1c","Type":"ContainerStarted","Data":"0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe"}
Jul 24 16:38:36.494977 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.494882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6" event={"ID":"354115bd-b27c-44cd-a27c-0c4d34692b1c","Type":"ContainerStarted","Data":"94ea6d7b5dbfab499638d1328f4d9ba8b6fa89a2dfbf5707dff382c6c541fbf9"}
Jul 24 16:38:36.494977 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.494903    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:36.509111 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.509075    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8" podStartSLOduration=1.509062674 podStartE2EDuration="1.509062674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:38:36.505768658 +0000 UTC m=+3098.833333207" watchObservedRunningTime="2026-07-24 16:38:36.509062674 +0000 UTC m=+3098.836627211"
Jul 24 16:38:36.520166 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.520122    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6" podStartSLOduration=1.520107734 podStartE2EDuration="1.520107734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:38:36.519225468 +0000 UTC m=+3098.846790005" watchObservedRunningTime="2026-07-24 16:38:36.520107734 +0000 UTC m=+3098.847672273"
Jul 24 16:38:36.951314 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:36.951285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:38:44.090214 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:44.090186    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:38:47.504105 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:47.504074    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:38:47.504569 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:47.504156    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:38:52.955116 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:52.955075    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"]
Jul 24 16:38:52.959007 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:52.958987    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:52.961077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:52.961059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:38:52.961187 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:52.961079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:38:52.965393 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:52.965367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"]
Jul 24 16:38:53.110668 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:53.110637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e0ac5c-4368-438a-86dc-7c6954503d3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb\" (UID: \"12e0ac5c-4368-438a-86dc-7c6954503d3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:53.211424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:53.211364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e0ac5c-4368-438a-86dc-7c6954503d3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb\" (UID: \"12e0ac5c-4368-438a-86dc-7c6954503d3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:53.211980 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:53.211961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e0ac5c-4368-438a-86dc-7c6954503d3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb\" (UID: \"12e0ac5c-4368-438a-86dc-7c6954503d3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:53.271481 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:53.271461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:53.403771 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:53.403747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"]
Jul 24 16:38:53.405125 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:38:53.405103    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e0ac5c_4368_438a_86dc_7c6954503d3f.slice/crio-fa2e9695c2da119ff628df3df86485399457ae83e45cf4c9152faa0b3c4c1eea WatchSource:0}: Error finding container fa2e9695c2da119ff628df3df86485399457ae83e45cf4c9152faa0b3c4c1eea: Status 404 returned error can't find the container with id fa2e9695c2da119ff628df3df86485399457ae83e45cf4c9152faa0b3c4c1eea
Jul 24 16:38:53.600045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:53.600017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb" event={"ID":"12e0ac5c-4368-438a-86dc-7c6954503d3f","Type":"ContainerStarted","Data":"fa2e9695c2da119ff628df3df86485399457ae83e45cf4c9152faa0b3c4c1eea"}
Jul 24 16:38:54.607243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:54.607207    2577 generic.go:356] "Generic (PLEG): container finished" podID="12e0ac5c-4368-438a-86dc-7c6954503d3f" containerID="38c5e952a467b43e4f03f746e61961a68825bbc14d8e6930fc5e222c9b911e64" exitCode=0
Jul 24 16:38:54.607603 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:54.607294    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb" event={"ID":"12e0ac5c-4368-438a-86dc-7c6954503d3f","Type":"ContainerDied","Data":"38c5e952a467b43e4f03f746e61961a68825bbc14d8e6930fc5e222c9b911e64"}
Jul 24 16:38:55.748471 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:55.748449    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:55.831689 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:55.831663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e0ac5c-4368-438a-86dc-7c6954503d3f-maas-api-service-ca\") pod \"12e0ac5c-4368-438a-86dc-7c6954503d3f\" (UID: \"12e0ac5c-4368-438a-86dc-7c6954503d3f\") "
Jul 24 16:38:55.831973 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:55.831954    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e0ac5c-4368-438a-86dc-7c6954503d3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12e0ac5c-4368-438a-86dc-7c6954503d3f" (UID: "12e0ac5c-4368-438a-86dc-7c6954503d3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:38:55.932806 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:55.932753    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e0ac5c-4368-438a-86dc-7c6954503d3f-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:38:56.619595 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:56.619562    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"
Jul 24 16:38:56.619779 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:56.619566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb" event={"ID":"12e0ac5c-4368-438a-86dc-7c6954503d3f","Type":"ContainerDied","Data":"fa2e9695c2da119ff628df3df86485399457ae83e45cf4c9152faa0b3c4c1eea"}
Jul 24 16:38:56.619779 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:38:56.619692    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa2e9695c2da119ff628df3df86485399457ae83e45cf4c9152faa0b3c4c1eea"
Jul 24 16:39:04.906427 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:04.906398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:04.977893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:04.977860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"]
Jul 24 16:39:04.978140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:04.978094    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8" podUID="20749917-6490-4cb3-a8da-6a864f4a91ac" containerName="payload-processing" containerID="cri-o://e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d" gracePeriod=30
Jul 24 16:39:05.001693 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.001667    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"]
Jul 24 16:39:05.001849 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.001832    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6" podUID="354115bd-b27c-44cd-a27c-0c4d34692b1c" containerName="payload-pre-processing" containerID="cri-o://0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe" gracePeriod=30
Jul 24 16:39:05.045735 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.045695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:05.120104 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.120072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:05.291045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.291022    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:39:05.294143 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.294125    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:39:05.405368 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.405342    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mblnr\" (UniqueName: \"kubernetes.io/projected/20749917-6490-4cb3-a8da-6a864f4a91ac-kube-api-access-mblnr\") pod \"20749917-6490-4cb3-a8da-6a864f4a91ac\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") "
Jul 24 16:39:05.405501 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.405400    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20749917-6490-4cb3-a8da-6a864f4a91ac-plugins-config-volume\") pod \"20749917-6490-4cb3-a8da-6a864f4a91ac\" (UID: \"20749917-6490-4cb3-a8da-6a864f4a91ac\") "
Jul 24 16:39:05.405501 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.405451    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354115bd-b27c-44cd-a27c-0c4d34692b1c-plugins-config-volume\") pod \"354115bd-b27c-44cd-a27c-0c4d34692b1c\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") "
Jul 24 16:39:05.405501 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.405487    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2dv9\" (UniqueName: \"kubernetes.io/projected/354115bd-b27c-44cd-a27c-0c4d34692b1c-kube-api-access-b2dv9\") pod \"354115bd-b27c-44cd-a27c-0c4d34692b1c\" (UID: \"354115bd-b27c-44cd-a27c-0c4d34692b1c\") "
Jul 24 16:39:05.405813 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.405790    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/354115bd-b27c-44cd-a27c-0c4d34692b1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "354115bd-b27c-44cd-a27c-0c4d34692b1c" (UID: "354115bd-b27c-44cd-a27c-0c4d34692b1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:39:05.405880 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.405820    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20749917-6490-4cb3-a8da-6a864f4a91ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20749917-6490-4cb3-a8da-6a864f4a91ac" (UID: "20749917-6490-4cb3-a8da-6a864f4a91ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:39:05.407452 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.407435    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354115bd-b27c-44cd-a27c-0c4d34692b1c-kube-api-access-b2dv9" (OuterVolumeSpecName: "kube-api-access-b2dv9") pod "354115bd-b27c-44cd-a27c-0c4d34692b1c" (UID: "354115bd-b27c-44cd-a27c-0c4d34692b1c"). InnerVolumeSpecName "kube-api-access-b2dv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:39:05.407507 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.407497    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20749917-6490-4cb3-a8da-6a864f4a91ac-kube-api-access-mblnr" (OuterVolumeSpecName: "kube-api-access-mblnr") pod "20749917-6490-4cb3-a8da-6a864f4a91ac" (UID: "20749917-6490-4cb3-a8da-6a864f4a91ac"). InnerVolumeSpecName "kube-api-access-mblnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:39:05.506012 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.505991    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2dv9\" (UniqueName: \"kubernetes.io/projected/354115bd-b27c-44cd-a27c-0c4d34692b1c-kube-api-access-b2dv9\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:05.506012 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.506013    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mblnr\" (UniqueName: \"kubernetes.io/projected/20749917-6490-4cb3-a8da-6a864f4a91ac-kube-api-access-mblnr\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:05.506161 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.506022    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20749917-6490-4cb3-a8da-6a864f4a91ac-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:05.506161 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.506031    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354115bd-b27c-44cd-a27c-0c4d34692b1c-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:05.671538 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.671445    2577 generic.go:356] "Generic (PLEG): container finished" podID="354115bd-b27c-44cd-a27c-0c4d34692b1c" containerID="0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe" exitCode=0
Jul 24 16:39:05.671538 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.671480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6" event={"ID":"354115bd-b27c-44cd-a27c-0c4d34692b1c","Type":"ContainerDied","Data":"0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe"}
Jul 24 16:39:05.671794 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.671494    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"
Jul 24 16:39:05.671794 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.671567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6" event={"ID":"354115bd-b27c-44cd-a27c-0c4d34692b1c","Type":"ContainerDied","Data":"94ea6d7b5dbfab499638d1328f4d9ba8b6fa89a2dfbf5707dff382c6c541fbf9"}
Jul 24 16:39:05.671794 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.671596    2577 scope.go:117] "RemoveContainer" containerID="0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe"
Jul 24 16:39:05.673045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.673017    2577 generic.go:356] "Generic (PLEG): container finished" podID="20749917-6490-4cb3-a8da-6a864f4a91ac" containerID="e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d" exitCode=0
Jul 24 16:39:05.673172 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.673054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8" event={"ID":"20749917-6490-4cb3-a8da-6a864f4a91ac","Type":"ContainerDied","Data":"e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d"}
Jul 24 16:39:05.673172 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.673090    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8" event={"ID":"20749917-6490-4cb3-a8da-6a864f4a91ac","Type":"ContainerDied","Data":"3d7417f46699caceea7f21bd2b00e413719924fe017c9ab1e9dc223ffd1d5c96"}
Jul 24 16:39:05.673172 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.673097    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"
Jul 24 16:39:05.684076 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.684051    2577 scope.go:117] "RemoveContainer" containerID="0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe"
Jul 24 16:39:05.684377 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:39:05.684355    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe\": container with ID starting with 0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe not found: ID does not exist" containerID="0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe"
Jul 24 16:39:05.684461 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.684383    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe"} err="failed to get container status \"0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe\": rpc error: code = NotFound desc = could not find container \"0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe\": container with ID starting with 0d3df555d51ca82b96563ccad07c261f50c13cac7f33546e32887ff5615c67fe not found: ID does not exist"
Jul 24 16:39:05.684461 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.684399    2577 scope.go:117] "RemoveContainer" containerID="e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d"
Jul 24 16:39:05.692988 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.692973    2577 scope.go:117] "RemoveContainer" containerID="e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d"
Jul 24 16:39:05.693238 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:39:05.693222    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d\": container with ID starting with e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d not found: ID does not exist" containerID="e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d"
Jul 24 16:39:05.693303 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.693248    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d"} err="failed to get container status \"e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d\": rpc error: code = NotFound desc = could not find container \"e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d\": container with ID starting with e8c5f5b78d62fe69725fdba75cdbeed84a99d0818bd0e382cc9bf577a2d23a5d not found: ID does not exist"
Jul 24 16:39:05.699312 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.699291    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"]
Jul 24 16:39:05.702652 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.702609    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-292d47-568cf4d644-qmrb6"]
Jul 24 16:39:05.710294 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.710273    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"]
Jul 24 16:39:05.717638 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:05.713898    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-292d47-85b54779f6-qkgh8"]
Jul 24 16:39:06.240059 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:06.240028    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20749917-6490-4cb3-a8da-6a864f4a91ac" path="/var/lib/kubelet/pods/20749917-6490-4cb3-a8da-6a864f4a91ac/volumes"
Jul 24 16:39:06.240396 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:06.240362    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354115bd-b27c-44cd-a27c-0c4d34692b1c" path="/var/lib/kubelet/pods/354115bd-b27c-44cd-a27c-0c4d34692b1c/volumes"
Jul 24 16:39:08.030819 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:08.030783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:08.032371 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:08.032348    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"]
Jul 24 16:39:08.035831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:08.035807    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-292d47-pb7gb"]
Jul 24 16:39:08.239384 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:08.239357    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e0ac5c-4368-438a-86dc-7c6954503d3f" path="/var/lib/kubelet/pods/12e0ac5c-4368-438a-86dc-7c6954503d3f/volumes"
Jul 24 16:39:08.402655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:08.402581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:09.894059 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:09.894029    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:39:14.291715 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:14.291676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:14.402958 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:14.402927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:14.571426 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:14.571344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:14.779315 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:14.779279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:18.091426 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091397    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"]
Jul 24 16:39:18.091858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091807    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e0ac5c-4368-438a-86dc-7c6954503d3f" containerName="revoke-keys"
Jul 24 16:39:18.091858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091821    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e0ac5c-4368-438a-86dc-7c6954503d3f" containerName="revoke-keys"
Jul 24 16:39:18.091858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091831    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354115bd-b27c-44cd-a27c-0c4d34692b1c" containerName="payload-pre-processing"
Jul 24 16:39:18.091858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091837    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="354115bd-b27c-44cd-a27c-0c4d34692b1c" containerName="payload-pre-processing"
Jul 24 16:39:18.091858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091851    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20749917-6490-4cb3-a8da-6a864f4a91ac" containerName="payload-processing"
Jul 24 16:39:18.091858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091856    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="20749917-6490-4cb3-a8da-6a864f4a91ac" containerName="payload-processing"
Jul 24 16:39:18.092074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091916    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="354115bd-b27c-44cd-a27c-0c4d34692b1c" containerName="payload-pre-processing"
Jul 24 16:39:18.092074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091924    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="20749917-6490-4cb3-a8da-6a864f4a91ac" containerName="payload-processing"
Jul 24 16:39:18.092074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.091934    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e0ac5c-4368-438a-86dc-7c6954503d3f" containerName="revoke-keys"
Jul 24 16:39:18.095128 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.095108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:18.096797 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.096778    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:39:18.096797 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.096789    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:39:18.101079 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.101057    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"]
Jul 24 16:39:18.211432 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.211403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359343ec-1308-423c-b15a-b2653fbe7737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq\" (UID: \"359343ec-1308-423c-b15a-b2653fbe7737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:18.311726 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.311698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359343ec-1308-423c-b15a-b2653fbe7737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq\" (UID: \"359343ec-1308-423c-b15a-b2653fbe7737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:18.312309 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.312289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359343ec-1308-423c-b15a-b2653fbe7737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq\" (UID: \"359343ec-1308-423c-b15a-b2653fbe7737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:18.408733 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.408673    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:18.536889 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.536858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"]
Jul 24 16:39:18.539348 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:39:18.539320    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod359343ec_1308_423c_b15a_b2653fbe7737.slice/crio-70a7107a94081ff0ccb03ec426dd316bd85f9f2c269ad5af81faba39d7dd79a7 WatchSource:0}: Error finding container 70a7107a94081ff0ccb03ec426dd316bd85f9f2c269ad5af81faba39d7dd79a7: Status 404 returned error can't find the container with id 70a7107a94081ff0ccb03ec426dd316bd85f9f2c269ad5af81faba39d7dd79a7
Jul 24 16:39:18.749156 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:18.749091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq" event={"ID":"359343ec-1308-423c-b15a-b2653fbe7737","Type":"ContainerStarted","Data":"70a7107a94081ff0ccb03ec426dd316bd85f9f2c269ad5af81faba39d7dd79a7"}
Jul 24 16:39:19.757074 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:19.757019    2577 generic.go:356] "Generic (PLEG): container finished" podID="359343ec-1308-423c-b15a-b2653fbe7737" containerID="e63ef493999ecbddf2a4369f515e6c3ea1eb67e7d3a828a42ac4368179016b5f" exitCode=0
Jul 24 16:39:19.757483 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:19.757234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq" event={"ID":"359343ec-1308-423c-b15a-b2653fbe7737","Type":"ContainerDied","Data":"e63ef493999ecbddf2a4369f515e6c3ea1eb67e7d3a828a42ac4368179016b5f"}
Jul 24 16:39:20.894509 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:20.894487    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:20.933607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:20.933578    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359343ec-1308-423c-b15a-b2653fbe7737-maas-api-service-ca\") pod \"359343ec-1308-423c-b15a-b2653fbe7737\" (UID: \"359343ec-1308-423c-b15a-b2653fbe7737\") "
Jul 24 16:39:20.933923 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:20.933899    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/359343ec-1308-423c-b15a-b2653fbe7737-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "359343ec-1308-423c-b15a-b2653fbe7737" (UID: "359343ec-1308-423c-b15a-b2653fbe7737"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:39:21.034740 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:21.034723    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359343ec-1308-423c-b15a-b2653fbe7737-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:21.769005 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:21.768977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq" event={"ID":"359343ec-1308-423c-b15a-b2653fbe7737","Type":"ContainerDied","Data":"70a7107a94081ff0ccb03ec426dd316bd85f9f2c269ad5af81faba39d7dd79a7"}
Jul 24 16:39:21.769156 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:21.769021    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70a7107a94081ff0ccb03ec426dd316bd85f9f2c269ad5af81faba39d7dd79a7"
Jul 24 16:39:21.769156 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:21.768985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"
Jul 24 16:39:35.607201 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:35.607166    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:39:36.860140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:36.860108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:36.897550 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:36.897523    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"]
Jul 24 16:39:36.897745 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:36.897723    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4" podUID="d6b621d9-b470-4a23-8792-1b09efe60ae7" containerName="payload-processing" containerID="cri-o://76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad" gracePeriod=30
Jul 24 16:39:36.905939 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:36.905714    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"]
Jul 24 16:39:36.905939 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:36.905907    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq" podUID="834cb701-e4c4-4f16-aafa-67f48494b7b8" containerName="payload-pre-processing" containerID="cri-o://d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54" gracePeriod=30
Jul 24 16:39:36.961045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:36.961008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:37.062590 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.062560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:37.199982 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.199957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:39:37.202723 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.202704    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:39:37.251464 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8zxx\" (UniqueName: \"kubernetes.io/projected/834cb701-e4c4-4f16-aafa-67f48494b7b8-kube-api-access-m8zxx\") pod \"834cb701-e4c4-4f16-aafa-67f48494b7b8\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") "
Jul 24 16:39:37.251464 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnmlr\" (UniqueName: \"kubernetes.io/projected/d6b621d9-b470-4a23-8792-1b09efe60ae7-kube-api-access-tnmlr\") pod \"d6b621d9-b470-4a23-8792-1b09efe60ae7\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") "
Jul 24 16:39:37.251678 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251501    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6b621d9-b470-4a23-8792-1b09efe60ae7-plugins-config-volume\") pod \"d6b621d9-b470-4a23-8792-1b09efe60ae7\" (UID: \"d6b621d9-b470-4a23-8792-1b09efe60ae7\") "
Jul 24 16:39:37.251678 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251519    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/834cb701-e4c4-4f16-aafa-67f48494b7b8-plugins-config-volume\") pod \"834cb701-e4c4-4f16-aafa-67f48494b7b8\" (UID: \"834cb701-e4c4-4f16-aafa-67f48494b7b8\") "
Jul 24 16:39:37.252094 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251848    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6b621d9-b470-4a23-8792-1b09efe60ae7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6b621d9-b470-4a23-8792-1b09efe60ae7" (UID: "d6b621d9-b470-4a23-8792-1b09efe60ae7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:39:37.252094 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251934    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834cb701-e4c4-4f16-aafa-67f48494b7b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "834cb701-e4c4-4f16-aafa-67f48494b7b8" (UID: "834cb701-e4c4-4f16-aafa-67f48494b7b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:39:37.252094 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.251941    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6b621d9-b470-4a23-8792-1b09efe60ae7-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:37.253649 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.253609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6b621d9-b470-4a23-8792-1b09efe60ae7-kube-api-access-tnmlr" (OuterVolumeSpecName: "kube-api-access-tnmlr") pod "d6b621d9-b470-4a23-8792-1b09efe60ae7" (UID: "d6b621d9-b470-4a23-8792-1b09efe60ae7"). InnerVolumeSpecName "kube-api-access-tnmlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:39:37.253917 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.253901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834cb701-e4c4-4f16-aafa-67f48494b7b8-kube-api-access-m8zxx" (OuterVolumeSpecName: "kube-api-access-m8zxx") pod "834cb701-e4c4-4f16-aafa-67f48494b7b8" (UID: "834cb701-e4c4-4f16-aafa-67f48494b7b8"). InnerVolumeSpecName "kube-api-access-m8zxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:39:37.352634 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.352596    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/834cb701-e4c4-4f16-aafa-67f48494b7b8-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:37.352724 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.352642    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8zxx\" (UniqueName: \"kubernetes.io/projected/834cb701-e4c4-4f16-aafa-67f48494b7b8-kube-api-access-m8zxx\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:37.352724 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.352652    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnmlr\" (UniqueName: \"kubernetes.io/projected/d6b621d9-b470-4a23-8792-1b09efe60ae7-kube-api-access-tnmlr\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:37.861432 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.861391    2577 generic.go:356] "Generic (PLEG): container finished" podID="834cb701-e4c4-4f16-aafa-67f48494b7b8" containerID="d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54" exitCode=0
Jul 24 16:39:37.861432 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.861430    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"
Jul 24 16:39:37.861954 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.861483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq" event={"ID":"834cb701-e4c4-4f16-aafa-67f48494b7b8","Type":"ContainerDied","Data":"d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54"}
Jul 24 16:39:37.861954 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.861533    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq" event={"ID":"834cb701-e4c4-4f16-aafa-67f48494b7b8","Type":"ContainerDied","Data":"cbe6f91181e914074638da4fcdccdadbfb33a2882a51d778cb2467da24420bf8"}
Jul 24 16:39:37.861954 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.861555    2577 scope.go:117] "RemoveContainer" containerID="d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54"
Jul 24 16:39:37.862941 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.862918    2577 generic.go:356] "Generic (PLEG): container finished" podID="d6b621d9-b470-4a23-8792-1b09efe60ae7" containerID="76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad" exitCode=0
Jul 24 16:39:37.863045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.862955    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"
Jul 24 16:39:37.863045 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.862998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4" event={"ID":"d6b621d9-b470-4a23-8792-1b09efe60ae7","Type":"ContainerDied","Data":"76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad"}
Jul 24 16:39:37.863126 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.863055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4" event={"ID":"d6b621d9-b470-4a23-8792-1b09efe60ae7","Type":"ContainerDied","Data":"9df215569667bfbcc753a42c034c93ffa635573fc77e95684f32b61f2039e2da"}
Jul 24 16:39:37.872032 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.872003    2577 scope.go:117] "RemoveContainer" containerID="d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54"
Jul 24 16:39:37.872279 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:39:37.872251    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54\": container with ID starting with d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54 not found: ID does not exist" containerID="d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54"
Jul 24 16:39:37.872372 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.872287    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54"} err="failed to get container status \"d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54\": rpc error: code = NotFound desc = could not find container \"d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54\": container with ID starting with d100960cba96a05b92a16a05903d7ed55c6896ebddb6a544ccdf7633a63b7a54 not found: ID does not exist"
Jul 24 16:39:37.872372 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.872309    2577 scope.go:117] "RemoveContainer" containerID="76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad"
Jul 24 16:39:37.880131 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.880117    2577 scope.go:117] "RemoveContainer" containerID="76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad"
Jul 24 16:39:37.880352 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:39:37.880336    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad\": container with ID starting with 76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad not found: ID does not exist" containerID="76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad"
Jul 24 16:39:37.880400 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.880360    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad"} err="failed to get container status \"76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad\": rpc error: code = NotFound desc = could not find container \"76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad\": container with ID starting with 76af84168a70ac0f76bd58ee9e17b336e170d56f979d37c368df0a1f8eafe8ad not found: ID does not exist"
Jul 24 16:39:37.887381 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.887360    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"]
Jul 24 16:39:37.891247 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.891223    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae4522-858cdfbb56-dvqxq"]
Jul 24 16:39:37.899268 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.899246    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"]
Jul 24 16:39:37.905140 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:37.905121    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae4522-588c59c6d8-rk5k4"]
Jul 24 16:39:38.168729 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.168641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:38.169891 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.169867    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"]
Jul 24 16:39:38.172189 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.172168    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ae4522-qvwnq"]
Jul 24 16:39:38.237988 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.237963    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="359343ec-1308-423c-b15a-b2653fbe7737" path="/var/lib/kubelet/pods/359343ec-1308-423c-b15a-b2653fbe7737/volumes"
Jul 24 16:39:38.238312 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.238299    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834cb701-e4c4-4f16-aafa-67f48494b7b8" path="/var/lib/kubelet/pods/834cb701-e4c4-4f16-aafa-67f48494b7b8/volumes"
Jul 24 16:39:38.238655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.238643    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6b621d9-b470-4a23-8792-1b09efe60ae7" path="/var/lib/kubelet/pods/d6b621d9-b470-4a23-8792-1b09efe60ae7/volumes"
Jul 24 16:39:38.531130 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:38.531106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:43.515010 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:43.514975    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"]
Jul 24 16:39:43.609746 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:43.609718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:39:46.091196 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:46.091162    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:39:58.534470 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.534419    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4" podUID="b5f285ab-a6b2-4f4d-8788-5b24656b2884" containerName="main" containerID="cri-o://0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe" gracePeriod=45
Jul 24 16:39:58.793026 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.792980    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:39:58.911317 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tls-certs\") pod \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") "
Jul 24 16:39:58.911457 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911351    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-home\") pod \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") "
Jul 24 16:39:58.911457 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911395    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-dshm\") pod \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") "
Jul 24 16:39:58.911457 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djwhl\" (UniqueName: \"kubernetes.io/projected/b5f285ab-a6b2-4f4d-8788-5b24656b2884-kube-api-access-djwhl\") pod \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") "
Jul 24 16:39:58.911610 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911459    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tmp-dir\") pod \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") "
Jul 24 16:39:58.911610 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911487    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-model-cache\") pod \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\" (UID: \"b5f285ab-a6b2-4f4d-8788-5b24656b2884\") "
Jul 24 16:39:58.911758 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911664    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-home" (OuterVolumeSpecName: "home") pod "b5f285ab-a6b2-4f4d-8788-5b24656b2884" (UID: "b5f285ab-a6b2-4f4d-8788-5b24656b2884"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:39:58.911824 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911795    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-home\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:58.911880 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911813    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5f285ab-a6b2-4f4d-8788-5b24656b2884" (UID: "b5f285ab-a6b2-4f4d-8788-5b24656b2884"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:39:58.911880 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.911835    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5f285ab-a6b2-4f4d-8788-5b24656b2884" (UID: "b5f285ab-a6b2-4f4d-8788-5b24656b2884"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:39:58.913575 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.913542    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-dshm" (OuterVolumeSpecName: "dshm") pod "b5f285ab-a6b2-4f4d-8788-5b24656b2884" (UID: "b5f285ab-a6b2-4f4d-8788-5b24656b2884"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:39:58.913726 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.913605    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5f285ab-a6b2-4f4d-8788-5b24656b2884" (UID: "b5f285ab-a6b2-4f4d-8788-5b24656b2884"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:39:58.913883 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.913859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f285ab-a6b2-4f4d-8788-5b24656b2884-kube-api-access-djwhl" (OuterVolumeSpecName: "kube-api-access-djwhl") pod "b5f285ab-a6b2-4f4d-8788-5b24656b2884" (UID: "b5f285ab-a6b2-4f4d-8788-5b24656b2884"). InnerVolumeSpecName "kube-api-access-djwhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:39:58.993414 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.993385    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5f285ab-a6b2-4f4d-8788-5b24656b2884" containerID="0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe" exitCode=0
Jul 24 16:39:58.993535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.993428    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"
Jul 24 16:39:58.993535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.993451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4" event={"ID":"b5f285ab-a6b2-4f4d-8788-5b24656b2884","Type":"ContainerDied","Data":"0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe"}
Jul 24 16:39:58.993535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.993480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4" event={"ID":"b5f285ab-a6b2-4f4d-8788-5b24656b2884","Type":"ContainerDied","Data":"d03c2d5704e2e3df2dffc1260c7837d891e71cadaaa29cf9a5cc1a40783fc18d"}
Jul 24 16:39:58.993535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:58.993497    2577 scope.go:117] "RemoveContainer" containerID="0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe"
Jul 24 16:39:59.002387 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.002361    2577 scope.go:117] "RemoveContainer" containerID="0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe"
Jul 24 16:39:59.002640 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:39:59.002597    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe\": container with ID starting with 0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe not found: ID does not exist" containerID="0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe"
Jul 24 16:39:59.002699 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.002637    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe"} err="failed to get container status \"0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe\": rpc error: code = NotFound desc = could not find container \"0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe\": container with ID starting with 0e7630a710d50d0df20a7566eb0ab8a6d55c17350f67b3be143d35c2358005fe not found: ID does not exist"
Jul 24 16:39:59.012766 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.012744    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tls-certs\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:59.012766 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.012763    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-dshm\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:59.012989 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.012773    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djwhl\" (UniqueName: \"kubernetes.io/projected/b5f285ab-a6b2-4f4d-8788-5b24656b2884-kube-api-access-djwhl\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:59.012989 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.012783    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-tmp-dir\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:59.012989 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.012793    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f285ab-a6b2-4f4d-8788-5b24656b2884-model-cache\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:39:59.014346 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.014323    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"]
Jul 24 16:39:59.022377 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:39:59.022358    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-ae4522/ipp-route-ae4522-kserve-5bc98f59c9-n62f4"]
Jul 24 16:40:00.237974 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:00.237938    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f285ab-a6b2-4f4d-8788-5b24656b2884" path="/var/lib/kubelet/pods/b5f285ab-a6b2-4f4d-8788-5b24656b2884/volumes"
Jul 24 16:40:20.839601 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:20.839565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:21.143491 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:21.143419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:27.067323 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:27.067289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:32.659535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:32.659504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:32.810576 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:32.810547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:32.934113 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:32.934023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:33.158518 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.158488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:33.274370 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274336    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"]
Jul 24 16:40:33.274796 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274779    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834cb701-e4c4-4f16-aafa-67f48494b7b8" containerName="payload-pre-processing"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274798    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="834cb701-e4c4-4f16-aafa-67f48494b7b8" containerName="payload-pre-processing"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274821    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="359343ec-1308-423c-b15a-b2653fbe7737" containerName="revoke-keys"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274829    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="359343ec-1308-423c-b15a-b2653fbe7737" containerName="revoke-keys"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274846    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6b621d9-b470-4a23-8792-1b09efe60ae7" containerName="payload-processing"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274855    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6b621d9-b470-4a23-8792-1b09efe60ae7" containerName="payload-processing"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274877    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f285ab-a6b2-4f4d-8788-5b24656b2884" containerName="main"
Jul 24 16:40:33.274893 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274885    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f285ab-a6b2-4f4d-8788-5b24656b2884" containerName="main"
Jul 24 16:40:33.275232 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274979    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="359343ec-1308-423c-b15a-b2653fbe7737" containerName="revoke-keys"
Jul 24 16:40:33.275232 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.274993    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6b621d9-b470-4a23-8792-1b09efe60ae7" containerName="payload-processing"
Jul 24 16:40:33.275232 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.275005    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="834cb701-e4c4-4f16-aafa-67f48494b7b8" containerName="payload-pre-processing"
Jul 24 16:40:33.275232 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.275014    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f285ab-a6b2-4f4d-8788-5b24656b2884" containerName="main"
Jul 24 16:40:33.278257 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.278238    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:33.280130 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.280112    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:40:33.280229 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.280139    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:40:33.285508 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.285484    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"]
Jul 24 16:40:33.365983 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.365952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce50df6-fba5-4749-996b-12260c4dcb15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6\" (UID: \"5ce50df6-fba5-4749-996b-12260c4dcb15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:33.466800 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.466770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce50df6-fba5-4749-996b-12260c4dcb15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6\" (UID: \"5ce50df6-fba5-4749-996b-12260c4dcb15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:33.467562 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.467544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce50df6-fba5-4749-996b-12260c4dcb15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6\" (UID: \"5ce50df6-fba5-4749-996b-12260c4dcb15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:33.591138 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.591045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:33.725684 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:33.725659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"]
Jul 24 16:40:33.727486 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:40:33.727459    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce50df6_fba5_4749_996b_12260c4dcb15.slice/crio-c9a6ee7e66441279468edcfa2bc1a310041171468eb3627cd5ca959b6f7224ef WatchSource:0}: Error finding container c9a6ee7e66441279468edcfa2bc1a310041171468eb3627cd5ca959b6f7224ef: Status 404 returned error can't find the container with id c9a6ee7e66441279468edcfa2bc1a310041171468eb3627cd5ca959b6f7224ef
Jul 24 16:40:34.205148 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:34.205052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6" event={"ID":"5ce50df6-fba5-4749-996b-12260c4dcb15","Type":"ContainerStarted","Data":"886e917c078de26ba33890f09234f0ea74a40d67b4b505886070f1608e28c2a0"}
Jul 24 16:40:34.205148 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:34.205102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6" event={"ID":"5ce50df6-fba5-4749-996b-12260c4dcb15","Type":"ContainerStarted","Data":"c9a6ee7e66441279468edcfa2bc1a310041171468eb3627cd5ca959b6f7224ef"}
Jul 24 16:40:34.218582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:34.218518    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6" podStartSLOduration=1.218501961 podStartE2EDuration="1.218501961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:40:34.217731682 +0000 UTC m=+3216.545296217" watchObservedRunningTime="2026-07-24 16:40:34.218501961 +0000 UTC m=+3216.546066498"
Jul 24 16:40:35.212646 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:35.212597    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ce50df6-fba5-4749-996b-12260c4dcb15" containerID="886e917c078de26ba33890f09234f0ea74a40d67b4b505886070f1608e28c2a0" exitCode=0
Jul 24 16:40:35.213076 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:35.212723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6" event={"ID":"5ce50df6-fba5-4749-996b-12260c4dcb15","Type":"ContainerDied","Data":"886e917c078de26ba33890f09234f0ea74a40d67b4b505886070f1608e28c2a0"}
Jul 24 16:40:36.353843 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:36.353822    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:36.493185 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:36.493108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce50df6-fba5-4749-996b-12260c4dcb15-maas-api-service-ca\") pod \"5ce50df6-fba5-4749-996b-12260c4dcb15\" (UID: \"5ce50df6-fba5-4749-996b-12260c4dcb15\") "
Jul 24 16:40:36.493438 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:36.493419    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce50df6-fba5-4749-996b-12260c4dcb15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ce50df6-fba5-4749-996b-12260c4dcb15" (UID: "5ce50df6-fba5-4749-996b-12260c4dcb15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:40:36.594581 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:36.594542    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce50df6-fba5-4749-996b-12260c4dcb15-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:40:37.225280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:37.225250    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"
Jul 24 16:40:37.225280 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:37.225265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6" event={"ID":"5ce50df6-fba5-4749-996b-12260c4dcb15","Type":"ContainerDied","Data":"c9a6ee7e66441279468edcfa2bc1a310041171468eb3627cd5ca959b6f7224ef"}
Jul 24 16:40:37.225492 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:37.225311    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9a6ee7e66441279468edcfa2bc1a310041171468eb3627cd5ca959b6f7224ef"
Jul 24 16:40:46.151451 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.151412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:46.182795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.182762    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"]
Jul 24 16:40:46.183037 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.182996    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv" podUID="c1138f6b-274d-4ca0-be59-b398b13a7184" containerName="payload-processing" containerID="cri-o://c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8" gracePeriod=30
Jul 24 16:40:46.191188 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.191163    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"]
Jul 24 16:40:46.191435 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.191411    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz" podUID="8b628a59-f0fa-4b74-beeb-9960d40f2c49" containerName="payload-pre-processing" containerID="cri-o://1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277" gracePeriod=30
Jul 24 16:40:46.221520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.221465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:46.278847 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.278819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:46.451234 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.451208    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:40:46.471964 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.471939    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:40:46.592367 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.592339    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b628a59-f0fa-4b74-beeb-9960d40f2c49-plugins-config-volume\") pod \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") "
Jul 24 16:40:46.592545 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.592441    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7klb\" (UniqueName: \"kubernetes.io/projected/8b628a59-f0fa-4b74-beeb-9960d40f2c49-kube-api-access-k7klb\") pod \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\" (UID: \"8b628a59-f0fa-4b74-beeb-9960d40f2c49\") "
Jul 24 16:40:46.592545 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.592464    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh2x9\" (UniqueName: \"kubernetes.io/projected/c1138f6b-274d-4ca0-be59-b398b13a7184-kube-api-access-dh2x9\") pod \"c1138f6b-274d-4ca0-be59-b398b13a7184\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") "
Jul 24 16:40:46.592545 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.592490    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1138f6b-274d-4ca0-be59-b398b13a7184-plugins-config-volume\") pod \"c1138f6b-274d-4ca0-be59-b398b13a7184\" (UID: \"c1138f6b-274d-4ca0-be59-b398b13a7184\") "
Jul 24 16:40:46.592911 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.592886    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b628a59-f0fa-4b74-beeb-9960d40f2c49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b628a59-f0fa-4b74-beeb-9960d40f2c49" (UID: "8b628a59-f0fa-4b74-beeb-9960d40f2c49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:40:46.592911 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.592900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1138f6b-274d-4ca0-be59-b398b13a7184-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1138f6b-274d-4ca0-be59-b398b13a7184" (UID: "c1138f6b-274d-4ca0-be59-b398b13a7184"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:40:46.594602 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.594577    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1138f6b-274d-4ca0-be59-b398b13a7184-kube-api-access-dh2x9" (OuterVolumeSpecName: "kube-api-access-dh2x9") pod "c1138f6b-274d-4ca0-be59-b398b13a7184" (UID: "c1138f6b-274d-4ca0-be59-b398b13a7184"). InnerVolumeSpecName "kube-api-access-dh2x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:40:46.594602 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.594589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b628a59-f0fa-4b74-beeb-9960d40f2c49-kube-api-access-k7klb" (OuterVolumeSpecName: "kube-api-access-k7klb") pod "8b628a59-f0fa-4b74-beeb-9960d40f2c49" (UID: "8b628a59-f0fa-4b74-beeb-9960d40f2c49"). InnerVolumeSpecName "kube-api-access-k7klb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:40:46.693582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.693502    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b628a59-f0fa-4b74-beeb-9960d40f2c49-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:40:46.693582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.693528    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7klb\" (UniqueName: \"kubernetes.io/projected/8b628a59-f0fa-4b74-beeb-9960d40f2c49-kube-api-access-k7klb\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:40:46.693582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.693539    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh2x9\" (UniqueName: \"kubernetes.io/projected/c1138f6b-274d-4ca0-be59-b398b13a7184-kube-api-access-dh2x9\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:40:46.693582 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:46.693547    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1138f6b-274d-4ca0-be59-b398b13a7184-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:40:47.279566 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.279531    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1138f6b-274d-4ca0-be59-b398b13a7184" containerID="c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8" exitCode=0
Jul 24 16:40:47.280055 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.279574    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"
Jul 24 16:40:47.280055 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.279632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv" event={"ID":"c1138f6b-274d-4ca0-be59-b398b13a7184","Type":"ContainerDied","Data":"c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8"}
Jul 24 16:40:47.280055 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.279685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv" event={"ID":"c1138f6b-274d-4ca0-be59-b398b13a7184","Type":"ContainerDied","Data":"389c54f9c88d4bed8bc81b3c8303c64e960ef90f4aa2124b3a245f4124a55699"}
Jul 24 16:40:47.280055 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.279711    2577 scope.go:117] "RemoveContainer" containerID="c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8"
Jul 24 16:40:47.281001 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.280985    2577 generic.go:356] "Generic (PLEG): container finished" podID="8b628a59-f0fa-4b74-beeb-9960d40f2c49" containerID="1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277" exitCode=0
Jul 24 16:40:47.281077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.281033    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"
Jul 24 16:40:47.281077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.281060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz" event={"ID":"8b628a59-f0fa-4b74-beeb-9960d40f2c49","Type":"ContainerDied","Data":"1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277"}
Jul 24 16:40:47.281181 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.281104    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz" event={"ID":"8b628a59-f0fa-4b74-beeb-9960d40f2c49","Type":"ContainerDied","Data":"2cc2c5ad7142a75119712a12900139ad3a7a0ba0b7a5548a95b429ed1d001b9c"}
Jul 24 16:40:47.288851 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.288832    2577 scope.go:117] "RemoveContainer" containerID="c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8"
Jul 24 16:40:47.289188 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:40:47.289088    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8\": container with ID starting with c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8 not found: ID does not exist" containerID="c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8"
Jul 24 16:40:47.289188 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.289119    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8"} err="failed to get container status \"c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8\": rpc error: code = NotFound desc = could not find container \"c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8\": container with ID starting with c18652e11e94a9ed6c1df68f763e998675f6b1f0a8268dce3e806f5f6eba57d8 not found: ID does not exist"
Jul 24 16:40:47.289188 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.289142    2577 scope.go:117] "RemoveContainer" containerID="1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277"
Jul 24 16:40:47.299291 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.299275    2577 scope.go:117] "RemoveContainer" containerID="1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277"
Jul 24 16:40:47.299542 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:40:47.299524    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277\": container with ID starting with 1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277 not found: ID does not exist" containerID="1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277"
Jul 24 16:40:47.299591 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.299550    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277"} err="failed to get container status \"1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277\": rpc error: code = NotFound desc = could not find container \"1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277\": container with ID starting with 1e4e58f717c47e8c889baf16d83fa6691d8cf2b18e88f6362dd26c022dbb3277 not found: ID does not exist"
Jul 24 16:40:47.306650 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.306626    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"]
Jul 24 16:40:47.309605 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.309586    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d71769-77dc7bdc56-x5ckv"]
Jul 24 16:40:47.318520 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.318503    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"]
Jul 24 16:40:47.323032 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:47.323014    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d71769-86448db8d4-xsjsz"]
Jul 24 16:40:48.237709 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:48.237678    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b628a59-f0fa-4b74-beeb-9960d40f2c49" path="/var/lib/kubelet/pods/8b628a59-f0fa-4b74-beeb-9960d40f2c49/volumes"
Jul 24 16:40:48.238043 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:48.238030    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1138f6b-274d-4ca0-be59-b398b13a7184" path="/var/lib/kubelet/pods/c1138f6b-274d-4ca0-be59-b398b13a7184/volumes"
Jul 24 16:40:48.511372 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:48.511280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:48.516640 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:48.516529    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"]
Jul 24 16:40:48.518979 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:48.518956    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d71769-pkpw6"]
Jul 24 16:40:48.882035 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:48.882001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:49.292357 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:49.292327    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:40:50.237888 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:50.237814    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce50df6-fba5-4749-996b-12260c4dcb15" path="/var/lib/kubelet/pods/5ce50df6-fba5-4749-996b-12260c4dcb15/volumes"
Jul 24 16:40:55.198790 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.198756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:55.300803 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.300774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:55.470948 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.470870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:55.699261 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.699223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:40:55.835654 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.835624    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"]
Jul 24 16:40:55.835970 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.835957    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b628a59-f0fa-4b74-beeb-9960d40f2c49" containerName="payload-pre-processing"
Jul 24 16:40:55.836016 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.835971    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b628a59-f0fa-4b74-beeb-9960d40f2c49" containerName="payload-pre-processing"
Jul 24 16:40:55.836016 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.835984    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1138f6b-274d-4ca0-be59-b398b13a7184" containerName="payload-processing"
Jul 24 16:40:55.836016 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.835989    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1138f6b-274d-4ca0-be59-b398b13a7184" containerName="payload-processing"
Jul 24 16:40:55.836016 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.836003    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce50df6-fba5-4749-996b-12260c4dcb15" containerName="revoke-keys"
Jul 24 16:40:55.836016 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.836009    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce50df6-fba5-4749-996b-12260c4dcb15" containerName="revoke-keys"
Jul 24 16:40:55.836195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.836067    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce50df6-fba5-4749-996b-12260c4dcb15" containerName="revoke-keys"
Jul 24 16:40:55.836195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.836077    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b628a59-f0fa-4b74-beeb-9960d40f2c49" containerName="payload-pre-processing"
Jul 24 16:40:55.836195 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.836084    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1138f6b-274d-4ca0-be59-b398b13a7184" containerName="payload-processing"
Jul 24 16:40:55.840560 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.840544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:55.842588 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.842570    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:40:55.842764 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.842739    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:40:55.846957 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.846920    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"]
Jul 24 16:40:55.866805 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.866775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e62e05-eb07-4b4d-bb66-5e89d587ef29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb\" (UID: \"54e62e05-eb07-4b4d-bb66-5e89d587ef29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:55.967141 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.967112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e62e05-eb07-4b4d-bb66-5e89d587ef29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb\" (UID: \"54e62e05-eb07-4b4d-bb66-5e89d587ef29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:55.967700 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:55.967683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e62e05-eb07-4b4d-bb66-5e89d587ef29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb\" (UID: \"54e62e05-eb07-4b4d-bb66-5e89d587ef29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:56.152318 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:56.152268    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:56.281293 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:56.281271    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"]
Jul 24 16:40:56.283015 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:40:56.282987    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e62e05_eb07_4b4d_bb66_5e89d587ef29.slice/crio-660c127e929c6084f8b03406774d93bb15f5390e90f16226b773a4f2d81be4a1 WatchSource:0}: Error finding container 660c127e929c6084f8b03406774d93bb15f5390e90f16226b773a4f2d81be4a1: Status 404 returned error can't find the container with id 660c127e929c6084f8b03406774d93bb15f5390e90f16226b773a4f2d81be4a1
Jul 24 16:40:56.332307 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:56.332277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb" event={"ID":"54e62e05-eb07-4b4d-bb66-5e89d587ef29","Type":"ContainerStarted","Data":"660c127e929c6084f8b03406774d93bb15f5390e90f16226b773a4f2d81be4a1"}
Jul 24 16:40:57.340036 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:57.340009    2577 generic.go:356] "Generic (PLEG): container finished" podID="54e62e05-eb07-4b4d-bb66-5e89d587ef29" containerID="019f4daf62c65b82aa3c5fbe91f29cc24c57c73ed454f6932171e65cfe70862d" exitCode=0
Jul 24 16:40:57.340392 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:57.340074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb" event={"ID":"54e62e05-eb07-4b4d-bb66-5e89d587ef29","Type":"ContainerDied","Data":"019f4daf62c65b82aa3c5fbe91f29cc24c57c73ed454f6932171e65cfe70862d"}
Jul 24 16:40:58.494252 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:58.494223    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:58.588527 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:58.588499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e62e05-eb07-4b4d-bb66-5e89d587ef29-maas-api-service-ca\") pod \"54e62e05-eb07-4b4d-bb66-5e89d587ef29\" (UID: \"54e62e05-eb07-4b4d-bb66-5e89d587ef29\") "
Jul 24 16:40:58.588829 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:58.588809    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54e62e05-eb07-4b4d-bb66-5e89d587ef29-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54e62e05-eb07-4b4d-bb66-5e89d587ef29" (UID: "54e62e05-eb07-4b4d-bb66-5e89d587ef29"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:40:58.689517 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:58.689459    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e62e05-eb07-4b4d-bb66-5e89d587ef29-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:40:59.352179 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:59.352151    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"
Jul 24 16:40:59.352322 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:59.352192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb" event={"ID":"54e62e05-eb07-4b4d-bb66-5e89d587ef29","Type":"ContainerDied","Data":"660c127e929c6084f8b03406774d93bb15f5390e90f16226b773a4f2d81be4a1"}
Jul 24 16:40:59.352322 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:40:59.352236    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="660c127e929c6084f8b03406774d93bb15f5390e90f16226b773a4f2d81be4a1"
Jul 24 16:41:02.435128 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:02.435086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:05.944441 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.944406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:05.945092 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.945070    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"]
Jul 24 16:41:05.948308 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.948283    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a5fa08-29748510-9l98g"]
Jul 24 16:41:05.975257 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.975232    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"]
Jul 24 16:41:05.975488 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.975466    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l" podUID="5118b6c4-b0b4-4dd6-8b40-d9814a279402" containerName="payload-processing" containerID="cri-o://24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b" gracePeriod=30
Jul 24 16:41:05.987529 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.987499    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"]
Jul 24 16:41:05.987736 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:05.987715    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd" podUID="d83819a0-3789-4143-844a-9b09aee4cdbf" containerName="payload-pre-processing" containerID="cri-o://f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286" gracePeriod=30
Jul 24 16:41:06.046170 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.046140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:06.144529 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.144499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:06.239275 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.239179    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41" path="/var/lib/kubelet/pods/91985fc2-3a73-4d4a-8eb4-3fbbe0ff6c41/volumes"
Jul 24 16:41:06.256627 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.256593    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:41:06.259263 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.259245    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:41:06.344821 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.344796    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q44t2\" (UniqueName: \"kubernetes.io/projected/d83819a0-3789-4143-844a-9b09aee4cdbf-kube-api-access-q44t2\") pod \"d83819a0-3789-4143-844a-9b09aee4cdbf\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") "
Jul 24 16:41:06.344955 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.344824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfk96\" (UniqueName: \"kubernetes.io/projected/5118b6c4-b0b4-4dd6-8b40-d9814a279402-kube-api-access-vfk96\") pod \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") "
Jul 24 16:41:06.344955 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.344853    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5118b6c4-b0b4-4dd6-8b40-d9814a279402-plugins-config-volume\") pod \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\" (UID: \"5118b6c4-b0b4-4dd6-8b40-d9814a279402\") "
Jul 24 16:41:06.344955 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.344883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d83819a0-3789-4143-844a-9b09aee4cdbf-plugins-config-volume\") pod \"d83819a0-3789-4143-844a-9b09aee4cdbf\" (UID: \"d83819a0-3789-4143-844a-9b09aee4cdbf\") "
Jul 24 16:41:06.345267 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.345241    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5118b6c4-b0b4-4dd6-8b40-d9814a279402-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5118b6c4-b0b4-4dd6-8b40-d9814a279402" (UID: "5118b6c4-b0b4-4dd6-8b40-d9814a279402"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:41:06.345341 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.345293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83819a0-3789-4143-844a-9b09aee4cdbf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d83819a0-3789-4143-844a-9b09aee4cdbf" (UID: "d83819a0-3789-4143-844a-9b09aee4cdbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:41:06.346898 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.346877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5118b6c4-b0b4-4dd6-8b40-d9814a279402-kube-api-access-vfk96" (OuterVolumeSpecName: "kube-api-access-vfk96") pod "5118b6c4-b0b4-4dd6-8b40-d9814a279402" (UID: "5118b6c4-b0b4-4dd6-8b40-d9814a279402"). InnerVolumeSpecName "kube-api-access-vfk96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:41:06.346956 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.346925    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d83819a0-3789-4143-844a-9b09aee4cdbf-kube-api-access-q44t2" (OuterVolumeSpecName: "kube-api-access-q44t2") pod "d83819a0-3789-4143-844a-9b09aee4cdbf" (UID: "d83819a0-3789-4143-844a-9b09aee4cdbf"). InnerVolumeSpecName "kube-api-access-q44t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:41:06.393858 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.393835    2577 generic.go:356] "Generic (PLEG): container finished" podID="5118b6c4-b0b4-4dd6-8b40-d9814a279402" containerID="24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b" exitCode=0
Jul 24 16:41:06.393945 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.393870    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"
Jul 24 16:41:06.393945 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.393915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l" event={"ID":"5118b6c4-b0b4-4dd6-8b40-d9814a279402","Type":"ContainerDied","Data":"24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b"}
Jul 24 16:41:06.394044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.393965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l" event={"ID":"5118b6c4-b0b4-4dd6-8b40-d9814a279402","Type":"ContainerDied","Data":"b8ed1e1d80b4321bafea19087436d355ed1ce10b29f929fe25907c42751d5427"}
Jul 24 16:41:06.394044 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.393996    2577 scope.go:117] "RemoveContainer" containerID="24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b"
Jul 24 16:41:06.395243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.395216    2577 generic.go:356] "Generic (PLEG): container finished" podID="d83819a0-3789-4143-844a-9b09aee4cdbf" containerID="f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286" exitCode=0
Jul 24 16:41:06.395243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.395234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd" event={"ID":"d83819a0-3789-4143-844a-9b09aee4cdbf","Type":"ContainerDied","Data":"f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286"}
Jul 24 16:41:06.395381 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.395255    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"
Jul 24 16:41:06.395381 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.395284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd" event={"ID":"d83819a0-3789-4143-844a-9b09aee4cdbf","Type":"ContainerDied","Data":"1f1533bcc536090ccff68a5cfe891b5bfeb3b7e8edf572eeb68b3d4781d083af"}
Jul 24 16:41:06.404359 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.404338    2577 scope.go:117] "RemoveContainer" containerID="24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b"
Jul 24 16:41:06.404597 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:41:06.404581    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b\": container with ID starting with 24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b not found: ID does not exist" containerID="24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b"
Jul 24 16:41:06.404694 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.404605    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b"} err="failed to get container status \"24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b\": rpc error: code = NotFound desc = could not find container \"24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b\": container with ID starting with 24a956eb62a696f86c5839e5ec5baa3094e16b1190a9b0c761450c0ba9ac595b not found: ID does not exist"
Jul 24 16:41:06.404694 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.404635    2577 scope.go:117] "RemoveContainer" containerID="f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286"
Jul 24 16:41:06.412805 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.412787    2577 scope.go:117] "RemoveContainer" containerID="f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286"
Jul 24 16:41:06.413035 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:41:06.413018    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286\": container with ID starting with f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286 not found: ID does not exist" containerID="f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286"
Jul 24 16:41:06.413078 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.413040    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286"} err="failed to get container status \"f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286\": rpc error: code = NotFound desc = could not find container \"f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286\": container with ID starting with f8c6a9e31493a6971e280444d5193039d8efdec95cda4dc8504264ba63828286 not found: ID does not exist"
Jul 24 16:41:06.417093 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.417073    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"]
Jul 24 16:41:06.422781 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.422759    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a5fa08-7cfcb45b6-h7x8l"]
Jul 24 16:41:06.431401 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.431379    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"]
Jul 24 16:41:06.433813 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.433794    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a5fa08-75f8f9d8-trjtd"]
Jul 24 16:41:06.445572 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.445552    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q44t2\" (UniqueName: \"kubernetes.io/projected/d83819a0-3789-4143-844a-9b09aee4cdbf-kube-api-access-q44t2\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:06.445672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.445573    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfk96\" (UniqueName: \"kubernetes.io/projected/5118b6c4-b0b4-4dd6-8b40-d9814a279402-kube-api-access-vfk96\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:06.445672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.445583    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5118b6c4-b0b4-4dd6-8b40-d9814a279402-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:06.445672 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:06.445593    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d83819a0-3789-4143-844a-9b09aee4cdbf-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:08.198230 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:08.198188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:41:08.238132 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:08.238103    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5118b6c4-b0b4-4dd6-8b40-d9814a279402" path="/var/lib/kubelet/pods/5118b6c4-b0b4-4dd6-8b40-d9814a279402/volumes"
Jul 24 16:41:08.238437 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:08.238424    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d83819a0-3789-4143-844a-9b09aee4cdbf" path="/var/lib/kubelet/pods/d83819a0-3789-4143-844a-9b09aee4cdbf/volumes"
Jul 24 16:41:10.905199 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:10.905162    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"]
Jul 24 16:41:10.905748 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:10.905554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:10.907306 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:10.907288    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a5fa08-v74pb"]
Jul 24 16:41:11.303648 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:11.303607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:12.237630 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:12.237595    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54e62e05-eb07-4b4d-bb66-5e89d587ef29" path="/var/lib/kubelet/pods/54e62e05-eb07-4b4d-bb66-5e89d587ef29/volumes"
Jul 24 16:41:17.888607 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:17.888569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:17.989859 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:17.989835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:18.147161 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.147081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:18.356490 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.356448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:18.468446 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468376    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"]
Jul 24 16:41:18.468751 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468738    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54e62e05-eb07-4b4d-bb66-5e89d587ef29" containerName="revoke-keys"
Jul 24 16:41:18.468795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468754    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="54e62e05-eb07-4b4d-bb66-5e89d587ef29" containerName="revoke-keys"
Jul 24 16:41:18.468795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468764    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5118b6c4-b0b4-4dd6-8b40-d9814a279402" containerName="payload-processing"
Jul 24 16:41:18.468795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468770    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5118b6c4-b0b4-4dd6-8b40-d9814a279402" containerName="payload-processing"
Jul 24 16:41:18.468795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468788    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83819a0-3789-4143-844a-9b09aee4cdbf" containerName="payload-pre-processing"
Jul 24 16:41:18.468795 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468794    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83819a0-3789-4143-844a-9b09aee4cdbf" containerName="payload-pre-processing"
Jul 24 16:41:18.468968 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468847    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d83819a0-3789-4143-844a-9b09aee4cdbf" containerName="payload-pre-processing"
Jul 24 16:41:18.468968 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468855    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="54e62e05-eb07-4b4d-bb66-5e89d587ef29" containerName="revoke-keys"
Jul 24 16:41:18.468968 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.468863    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5118b6c4-b0b4-4dd6-8b40-d9814a279402" containerName="payload-processing"
Jul 24 16:41:18.472037 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.472022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:18.474005 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.473982    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:41:18.474119 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.474031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hchgr\""
Jul 24 16:41:18.479716 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.479692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"]
Jul 24 16:41:18.532502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.532480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t\" (UID: \"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:18.633152 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.633130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t\" (UID: \"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:18.633729 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.633713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t\" (UID: \"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:18.784471 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.784452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:18.917036 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:18.917005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"]
Jul 24 16:41:18.918290 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:41:18.918265    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e98016c_60c3_47d5_b3d7_35d8ca9c2f58.slice/crio-3aaac2fe6a0d9d6e599806fc643238f89e6c01bf8f3217041d62930ade251972 WatchSource:0}: Error finding container 3aaac2fe6a0d9d6e599806fc643238f89e6c01bf8f3217041d62930ade251972: Status 404 returned error can't find the container with id 3aaac2fe6a0d9d6e599806fc643238f89e6c01bf8f3217041d62930ade251972
Jul 24 16:41:19.471243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:19.471170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t" event={"ID":"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58","Type":"ContainerStarted","Data":"30fd160d35f38da1e71c8d591230553dad14d85406381ba7abb8a1929bc7b6e3"}
Jul 24 16:41:19.471243 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:19.471231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t" event={"ID":"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58","Type":"ContainerStarted","Data":"3aaac2fe6a0d9d6e599806fc643238f89e6c01bf8f3217041d62930ade251972"}
Jul 24 16:41:19.484712 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:19.484669    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t" podStartSLOduration=1.484655667 podStartE2EDuration="1.484655667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:41:19.483016111 +0000 UTC m=+3261.810580848" watchObservedRunningTime="2026-07-24 16:41:19.484655667 +0000 UTC m=+3261.812220206"
Jul 24 16:41:20.478261 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:20.478224    2577 generic.go:356] "Generic (PLEG): container finished" podID="0e98016c-60c3-47d5-b3d7-35d8ca9c2f58" containerID="30fd160d35f38da1e71c8d591230553dad14d85406381ba7abb8a1929bc7b6e3" exitCode=0
Jul 24 16:41:20.478649 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:20.478306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t" event={"ID":"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58","Type":"ContainerDied","Data":"30fd160d35f38da1e71c8d591230553dad14d85406381ba7abb8a1929bc7b6e3"}
Jul 24 16:41:21.608636 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:21.608602    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:21.655455 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:21.655427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58-maas-api-service-ca\") pod \"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58\" (UID: \"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58\") "
Jul 24 16:41:21.655769 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:21.655748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e98016c-60c3-47d5-b3d7-35d8ca9c2f58" (UID: "0e98016c-60c3-47d5-b3d7-35d8ca9c2f58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:41:21.756449 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:21.756395    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58-maas-api-service-ca\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:22.490181 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:22.490149    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"
Jul 24 16:41:22.490348 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:22.490148    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t" event={"ID":"0e98016c-60c3-47d5-b3d7-35d8ca9c2f58","Type":"ContainerDied","Data":"3aaac2fe6a0d9d6e599806fc643238f89e6c01bf8f3217041d62930ade251972"}
Jul 24 16:41:22.490348 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:22.490261    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aaac2fe6a0d9d6e599806fc643238f89e6c01bf8f3217041d62930ade251972"
Jul 24 16:41:29.576371 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.576104    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:29.580822 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.580779    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"]
Jul 24 16:41:29.582913 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.582891    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-675edf-29748510-k8csq"]
Jul 24 16:41:29.603116 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.603092    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"]
Jul 24 16:41:29.603335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.603295    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh" podUID="d49e908d-8827-4f36-86f6-b02333476718" containerName="payload-processing" containerID="cri-o://f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c" gracePeriod=30
Jul 24 16:41:29.614502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.614479    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"]
Jul 24 16:41:29.614955 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.614897    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" podUID="e98bba96-c386-49a2-b340-df57d150a72f" containerName="payload-pre-processing" containerID="cri-o://89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c" gracePeriod=30
Jul 24 16:41:29.649197 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.649174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:29.706977 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.706956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:29.890153 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.890130    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:41:29.893048 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:29.893030    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:41:30.019143 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.019115    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkkdv\" (UniqueName: \"kubernetes.io/projected/e98bba96-c386-49a2-b340-df57d150a72f-kube-api-access-gkkdv\") pod \"e98bba96-c386-49a2-b340-df57d150a72f\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") "
Jul 24 16:41:30.019302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.019162    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49e908d-8827-4f36-86f6-b02333476718-plugins-config-volume\") pod \"d49e908d-8827-4f36-86f6-b02333476718\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") "
Jul 24 16:41:30.019302 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.019182    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2t7v\" (UniqueName: \"kubernetes.io/projected/d49e908d-8827-4f36-86f6-b02333476718-kube-api-access-g2t7v\") pod \"d49e908d-8827-4f36-86f6-b02333476718\" (UID: \"d49e908d-8827-4f36-86f6-b02333476718\") "
Jul 24 16:41:30.019499 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.019376    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e98bba96-c386-49a2-b340-df57d150a72f-plugins-config-volume\") pod \"e98bba96-c386-49a2-b340-df57d150a72f\" (UID: \"e98bba96-c386-49a2-b340-df57d150a72f\") "
Jul 24 16:41:30.019635 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.019584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d49e908d-8827-4f36-86f6-b02333476718-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d49e908d-8827-4f36-86f6-b02333476718" (UID: "d49e908d-8827-4f36-86f6-b02333476718"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:41:30.019758 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.019700    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e98bba96-c386-49a2-b340-df57d150a72f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e98bba96-c386-49a2-b340-df57d150a72f" (UID: "e98bba96-c386-49a2-b340-df57d150a72f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:41:30.021251 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.021229    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49e908d-8827-4f36-86f6-b02333476718-kube-api-access-g2t7v" (OuterVolumeSpecName: "kube-api-access-g2t7v") pod "d49e908d-8827-4f36-86f6-b02333476718" (UID: "d49e908d-8827-4f36-86f6-b02333476718"). InnerVolumeSpecName "kube-api-access-g2t7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:41:30.021335 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.021258    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98bba96-c386-49a2-b340-df57d150a72f-kube-api-access-gkkdv" (OuterVolumeSpecName: "kube-api-access-gkkdv") pod "e98bba96-c386-49a2-b340-df57d150a72f" (UID: "e98bba96-c386-49a2-b340-df57d150a72f"). InnerVolumeSpecName "kube-api-access-gkkdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:41:30.120522 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.120473    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49e908d-8827-4f36-86f6-b02333476718-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:30.120522 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.120492    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2t7v\" (UniqueName: \"kubernetes.io/projected/d49e908d-8827-4f36-86f6-b02333476718-kube-api-access-g2t7v\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:30.120522 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.120501    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e98bba96-c386-49a2-b340-df57d150a72f-plugins-config-volume\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:30.120522 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.120510    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkkdv\" (UniqueName: \"kubernetes.io/projected/e98bba96-c386-49a2-b340-df57d150a72f-kube-api-access-gkkdv\") on node \"ip-10-0-133-118.ec2.internal\" DevicePath \"\""
Jul 24 16:41:30.238572 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.238543    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5" path="/var/lib/kubelet/pods/b86bd3c6-d5b0-4928-bffb-f4ddbc13abf5/volumes"
Jul 24 16:41:30.535159 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.535132    2577 generic.go:356] "Generic (PLEG): container finished" podID="d49e908d-8827-4f36-86f6-b02333476718" containerID="f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c" exitCode=0
Jul 24 16:41:30.535278 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.535176    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"
Jul 24 16:41:30.535278 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.535220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh" event={"ID":"d49e908d-8827-4f36-86f6-b02333476718","Type":"ContainerDied","Data":"f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c"}
Jul 24 16:41:30.535278 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.535269    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh" event={"ID":"d49e908d-8827-4f36-86f6-b02333476718","Type":"ContainerDied","Data":"5086393c2d24e29e267c999716d03ef443ae4351980118cbd0fa7287036aa13c"}
Jul 24 16:41:30.535439 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.535300    2577 scope.go:117] "RemoveContainer" containerID="f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c"
Jul 24 16:41:30.536593 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.536570    2577 generic.go:356] "Generic (PLEG): container finished" podID="e98bba96-c386-49a2-b340-df57d150a72f" containerID="89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c" exitCode=0
Jul 24 16:41:30.536722 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.536610    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"
Jul 24 16:41:30.536722 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.536657    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" event={"ID":"e98bba96-c386-49a2-b340-df57d150a72f","Type":"ContainerDied","Data":"89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c"}
Jul 24 16:41:30.536722 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.536709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs" event={"ID":"e98bba96-c386-49a2-b340-df57d150a72f","Type":"ContainerDied","Data":"cfb0ee562f1080f915fe636055909f9d30235bdb1e124d91d48540e57be74a25"}
Jul 24 16:41:30.544424 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.544409    2577 scope.go:117] "RemoveContainer" containerID="f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c"
Jul 24 16:41:30.544726 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:41:30.544709    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c\": container with ID starting with f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c not found: ID does not exist" containerID="f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c"
Jul 24 16:41:30.544780 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.544731    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c"} err="failed to get container status \"f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c\": rpc error: code = NotFound desc = could not find container \"f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c\": container with ID starting with f4c078a8f169c6beb5130dc83c9cc3c6140b1aaf6e7c78ea4653ab95ba55931c not found: ID does not exist"
Jul 24 16:41:30.544780 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.544748    2577 scope.go:117] "RemoveContainer" containerID="89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c"
Jul 24 16:41:30.551363 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.551331    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"]
Jul 24 16:41:30.553220 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.553204    2577 scope.go:117] "RemoveContainer" containerID="89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c"
Jul 24 16:41:30.553493 ip-10-0-133-118 kubenswrapper[2577]: E0724 16:41:30.553474    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c\": container with ID starting with 89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c not found: ID does not exist" containerID="89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c"
Jul 24 16:41:30.553563 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.553502    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c"} err="failed to get container status \"89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c\": rpc error: code = NotFound desc = could not find container \"89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c\": container with ID starting with 89ae102b9208aaa281a04d6fd20eeb3201229ff6c1061fb116f4f1f8b65ee17c not found: ID does not exist"
Jul 24 16:41:30.554756 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.554736    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-675edf-546749f5cd-whxhh"]
Jul 24 16:41:30.561914 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.561895    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"]
Jul 24 16:41:30.565683 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:30.565667    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-675edf-cc4f9444-wj4zs"]
Jul 24 16:41:32.238265 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:32.238230    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49e908d-8827-4f36-86f6-b02333476718" path="/var/lib/kubelet/pods/d49e908d-8827-4f36-86f6-b02333476718/volumes"
Jul 24 16:41:32.238662 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:32.238562    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e98bba96-c386-49a2-b340-df57d150a72f" path="/var/lib/kubelet/pods/e98bba96-c386-49a2-b340-df57d150a72f/volumes"
Jul 24 16:41:32.488097 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:32.488069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-765wv"]
Jul 24 16:41:33.536525 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:33.536484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:33.537017 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:33.536641    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"]
Jul 24 16:41:33.538737 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:33.538717    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-675edf-4wp4t"]
Jul 24 16:41:33.894260 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:33.894171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:34.238263 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:34.238194    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e98016c-60c3-47d5-b3d7-35d8ca9c2f58" path="/var/lib/kubelet/pods/0e98016c-60c3-47d5-b3d7-35d8ca9c2f58/volumes"
Jul 24 16:41:39.859251 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:39.859216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:39.960042 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:39.960005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:40.203441 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:40.203323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:40.371365 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:40.371331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:46.534077 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:46.534048    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749946f7d7-c4ps8_08737d3f-3b02-4c1e-a82d-c70113b2f206/manager/0.log"
Jul 24 16:41:46.540190 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:46.540171    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749946f7d7-c4ps8_08737d3f-3b02-4c1e-a82d-c70113b2f206/copy-manifests/0.log"
Jul 24 16:41:47.221411 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:47.221382    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748480-66xwd_c34d5d44-f892-4b8e-97a8-211d23284e88/cleanup/0.log"
Jul 24 16:41:47.326098 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:47.326075    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748495-7hxdx_14b3561d-27ab-4954-9576-fd1aa137f077/cleanup/0.log"
Jul 24 16:41:47.439553 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:47.439514    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748510-r5q9v_80737204-2412-4455-9b04-e4bdf5d0843e/cleanup/0.log"
Jul 24 16:41:48.285141 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.285116    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48_59c12460-bbc3-4f82-8724-5fc8804b2425/extract/0.log"
Jul 24 16:41:48.290895 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.290871    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48_59c12460-bbc3-4f82-8724-5fc8804b2425/util/0.log"
Jul 24 16:41:48.296506 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.296490    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48_59c12460-bbc3-4f82-8724-5fc8804b2425/pull/0.log"
Jul 24 16:41:48.404491 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.404469    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg_d939ea3b-bb02-475e-abd0-70d5f750bb56/util/0.log"
Jul 24 16:41:48.411042 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.411013    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg_d939ea3b-bb02-475e-abd0-70d5f750bb56/pull/0.log"
Jul 24 16:41:48.417457 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.417440    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg_d939ea3b-bb02-475e-abd0-70d5f750bb56/extract/0.log"
Jul 24 16:41:48.527129 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.527112    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4_481a9218-3188-404e-8ce8-39e0958151e9/extract/0.log"
Jul 24 16:41:48.533769 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.533752    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4_481a9218-3188-404e-8ce8-39e0958151e9/util/0.log"
Jul 24 16:41:48.540822 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.540772    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4_481a9218-3188-404e-8ce8-39e0958151e9/pull/0.log"
Jul 24 16:41:48.650732 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.650715    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m_c3ea0ec8-3bac-4d4e-8453-206ac8fe1209/pull/0.log"
Jul 24 16:41:48.656854 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.656839    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m_c3ea0ec8-3bac-4d4e-8453-206ac8fe1209/extract/0.log"
Jul 24 16:41:48.662641 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:48.662608    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m_c3ea0ec8-3bac-4d4e-8453-206ac8fe1209/util/0.log"
Jul 24 16:41:49.448147 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:49.448122    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-765wv_f3a307db-b80c-4bff-95eb-0cfd6926b05e/limitador/0.log"
Jul 24 16:41:50.045998 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:50.045965    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-f8fhw_f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e/discovery/0.log"
Jul 24 16:41:50.736034 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:50.736008    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-747d9dcfc7-cs2nw_e1a4672e-a764-4839-82de-f0a45a3ed786/router/0.log"
Jul 24 16:41:51.404109 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:51.404080    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v7tc_011d7423-df6a-49f1-beb2-5084e1fe4419/main/0.log"
Jul 24 16:41:56.824933 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:56.824888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:56.925759 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:56.925727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log" limit=1024
Jul 24 16:41:58.783462 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:58.783431    2577 scope.go:117] "RemoveContainer" containerID="f4dbba64e35da777200557a9b2a6903eaea261c39339795e7548de006f962856"
Jul 24 16:41:58.791635 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:58.791603    2577 scope.go:117] "RemoveContainer" containerID="f4e07c32977ba3cbc83c179db375c6daef3e9bbb0ff4a46fe9268e5ce823db55"
Jul 24 16:41:59.548721 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:59.548695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lprwc_95304c49-01ee-4b56-8b45-e33b3f2d5b39/global-pull-secret-syncer/0.log"
Jul 24 16:41:59.633072 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:59.633052    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-49nk2_2af47af0-b016-4f57-acc9-918904ff8903/konnectivity-agent/0.log"
Jul 24 16:41:59.729772 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:41:59.729744    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-118.ec2.internal_7888cd199d1af8b29d9e50b2c19015c9/haproxy/0.log"
Jul 24 16:42:04.079844 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.079813    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48_59c12460-bbc3-4f82-8724-5fc8804b2425/extract/0.log"
Jul 24 16:42:04.105581 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.105558    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48_59c12460-bbc3-4f82-8724-5fc8804b2425/util/0.log"
Jul 24 16:42:04.129089 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.129073    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vsn48_59c12460-bbc3-4f82-8724-5fc8804b2425/pull/0.log"
Jul 24 16:42:04.154158 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.154141    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg_d939ea3b-bb02-475e-abd0-70d5f750bb56/extract/0.log"
Jul 24 16:42:04.174450 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.174434    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg_d939ea3b-bb02-475e-abd0-70d5f750bb56/util/0.log"
Jul 24 16:42:04.195942 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.195919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e049cmg_d939ea3b-bb02-475e-abd0-70d5f750bb56/pull/0.log"
Jul 24 16:42:04.222963 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.222948    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4_481a9218-3188-404e-8ce8-39e0958151e9/extract/0.log"
Jul 24 16:42:04.243492 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.243470    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4_481a9218-3188-404e-8ce8-39e0958151e9/util/0.log"
Jul 24 16:42:04.264092 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.264077    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bztk4_481a9218-3188-404e-8ce8-39e0958151e9/pull/0.log"
Jul 24 16:42:04.291263 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.291241    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m_c3ea0ec8-3bac-4d4e-8453-206ac8fe1209/extract/0.log"
Jul 24 16:42:04.320380 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.320356    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m_c3ea0ec8-3bac-4d4e-8453-206ac8fe1209/util/0.log"
Jul 24 16:42:04.339176 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:04.339119    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lw46m_c3ea0ec8-3bac-4d4e-8453-206ac8fe1209/pull/0.log"
Jul 24 16:42:05.033867 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:05.033842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-765wv_f3a307db-b80c-4bff-95eb-0cfd6926b05e/limitador/0.log"
Jul 24 16:42:07.027857 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.027828    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-cdps2_e3cfeeb6-3cf6-415c-85cf-71e495eca893/cluster-monitoring-operator/0.log"
Jul 24 16:42:07.065528 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.065506    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-bt4wc_cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91/kube-state-metrics/0.log"
Jul 24 16:42:07.087040 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.087022    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-bt4wc_cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91/kube-rbac-proxy-main/0.log"
Jul 24 16:42:07.115921 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.115905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-bt4wc_cca9eae0-0a2b-4f7c-8c9c-2f1271bd6d91/kube-rbac-proxy-self/0.log"
Jul 24 16:42:07.170655 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.170634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-6ckzh_1bf748a1-0011-4055-ad7c-e9b8c8a8deb3/monitoring-plugin/0.log"
Jul 24 16:42:07.293316 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.293264    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rhxcd_690f9ba2-c7a6-464a-a550-81526d637617/node-exporter/0.log"
Jul 24 16:42:07.314367 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.314348    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rhxcd_690f9ba2-c7a6-464a-a550-81526d637617/kube-rbac-proxy/0.log"
Jul 24 16:42:07.334016 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.333999    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rhxcd_690f9ba2-c7a6-464a-a550-81526d637617/init-textfile/0.log"
Jul 24 16:42:07.441880 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.441862    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-54tdb_b76a2700-72f3-49f6-b4b6-3c676a126e72/kube-rbac-proxy-main/0.log"
Jul 24 16:42:07.462869 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.462850    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-54tdb_b76a2700-72f3-49f6-b4b6-3c676a126e72/kube-rbac-proxy-self/0.log"
Jul 24 16:42:07.483505 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.483483    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-54tdb_b76a2700-72f3-49f6-b4b6-3c676a126e72/openshift-state-metrics/0.log"
Jul 24 16:42:07.684741 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.684688    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-bcnsq_1af98115-d18d-4479-a87d-cc8b38c5ef41/prometheus-operator/0.log"
Jul 24 16:42:07.702496 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:07.702474    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-bcnsq_1af98115-d18d-4479-a87d-cc8b38c5ef41/kube-rbac-proxy/0.log"
Jul 24 16:42:09.140677 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.140640    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-ndwsn_f69a5ff3-fda7-4f1d-bf17-88f8f8085399/networking-console-plugin/0.log"
Jul 24 16:42:09.583651 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.583607    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"]
Jul 24 16:42:09.584354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584278    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e98bba96-c386-49a2-b340-df57d150a72f" containerName="payload-pre-processing"
Jul 24 16:42:09.584354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584302    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e98bba96-c386-49a2-b340-df57d150a72f" containerName="payload-pre-processing"
Jul 24 16:42:09.584354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584326    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e98016c-60c3-47d5-b3d7-35d8ca9c2f58" containerName="revoke-keys"
Jul 24 16:42:09.584354 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584336    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e98016c-60c3-47d5-b3d7-35d8ca9c2f58" containerName="revoke-keys"
Jul 24 16:42:09.584695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584367    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49e908d-8827-4f36-86f6-b02333476718" containerName="payload-processing"
Jul 24 16:42:09.584695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584377    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49e908d-8827-4f36-86f6-b02333476718" containerName="payload-processing"
Jul 24 16:42:09.584695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584457    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e98bba96-c386-49a2-b340-df57d150a72f" containerName="payload-pre-processing"
Jul 24 16:42:09.584695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584469    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49e908d-8827-4f36-86f6-b02333476718" containerName="payload-processing"
Jul 24 16:42:09.584695 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.584477    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e98016c-60c3-47d5-b3d7-35d8ca9c2f58" containerName="revoke-keys"
Jul 24 16:42:09.589904 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.589883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.591674 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.591656    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6ww8s\"/\"default-dockercfg-t8xkf\""
Jul 24 16:42:09.591778 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.591660    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6ww8s\"/\"openshift-service-ca.crt\""
Jul 24 16:42:09.592588 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.592418    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6ww8s\"/\"kube-root-ca.crt\""
Jul 24 16:42:09.596577 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.596555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"]
Jul 24 16:42:09.699894 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.699875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-proc\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.700001 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.699907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-sys\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.700001 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.699934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfw77\" (UniqueName: \"kubernetes.io/projected/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-kube-api-access-mfw77\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.700001 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.699974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-podres\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.700110 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.700004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-lib-modules\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.800894 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.800861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-proc\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.800997 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.800905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-sys\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.800997 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.800954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfw77\" (UniqueName: \"kubernetes.io/projected/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-kube-api-access-mfw77\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.800997 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.800968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-sys\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.800997 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.800986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-podres\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.801136 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.801012    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-proc\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.801136 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.801018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-lib-modules\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.801205 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.801143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-lib-modules\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.801205 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.801143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-podres\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.807897 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.807877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfw77\" (UniqueName: \"kubernetes.io/projected/ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d-kube-api-access-mfw77\") pod \"perf-node-gather-daemonset-4z5jb\" (UID: \"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d\") " pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:09.901535 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:09.901478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:10.023896 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.023874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"]
Jul 24 16:42:10.026083 ip-10-0-133-118 kubenswrapper[2577]: W0724 16:42:10.026050    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podffda2cbf_1ebb_4e2d_a94e_99f05ad96a5d.slice/crio-4dc6442c63fc654a669ab68cc2e9dc5381a589e1419c02227ce77f1cf07bb6c3 WatchSource:0}: Error finding container 4dc6442c63fc654a669ab68cc2e9dc5381a589e1419c02227ce77f1cf07bb6c3: Status 404 returned error can't find the container with id 4dc6442c63fc654a669ab68cc2e9dc5381a589e1419c02227ce77f1cf07bb6c3
Jul 24 16:42:10.315122 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.315099    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c645cc5b8-2w6j4_0d3269b8-8c39-4d05-8987-d199e1324436/console/0.log"
Jul 24 16:42:10.364631 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.364599    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-5vjb8_73956ad7-24a7-40f8-9fa3-40900fc523f8/download-server/0.log"
Jul 24 16:42:10.768743 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.768668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb" event={"ID":"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d","Type":"ContainerStarted","Data":"9bb0c260b11eb99e3d4137ae50e80d964bb631677507eb526bf767351ed68236"}
Jul 24 16:42:10.768743 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.768718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb" event={"ID":"ffda2cbf-1ebb-4e2d-a94e-99f05ad96a5d","Type":"ContainerStarted","Data":"4dc6442c63fc654a669ab68cc2e9dc5381a589e1419c02227ce77f1cf07bb6c3"}
Jul 24 16:42:10.768931 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.768758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:10.782633 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:10.782571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb" podStartSLOduration=1.7825592239999999 podStartE2EDuration="1.782559224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:42:10.781576654 +0000 UTC m=+3313.109141192" watchObservedRunningTime="2026-07-24 16:42:10.782559224 +0000 UTC m=+3313.110123761"
Jul 24 16:42:12.090793 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:12.090762    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vpr5g_14350b95-4ee8-4311-9bf0-f26139c5ca3e/dns/0.log"
Jul 24 16:42:12.110914 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:12.110869    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vpr5g_14350b95-4ee8-4311-9bf0-f26139c5ca3e/kube-rbac-proxy/0.log"
Jul 24 16:42:12.133352 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:12.133333    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b2tj6_8105f600-365c-42f2-bd83-2dc9190a4870/dns-node-resolver/0.log"
Jul 24 16:42:12.789422 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:12.789393    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lbzkd_9043edc6-8df3-47b5-9318-b7a478366ec3/node-ca/0.log"
Jul 24 16:42:13.835713 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:13.835684    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-f8fhw_f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e/discovery/0.log"
Jul 24 16:42:14.025464 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:14.025438    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-747d9dcfc7-cs2nw_e1a4672e-a764-4839-82de-f0a45a3ed786/router/0.log"
Jul 24 16:42:14.845226 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:14.845198    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zmcqg_1690e434-d4c8-40f3-a5bf-0a20714c779a/serve-healthcheck-canary/0.log"
Jul 24 16:42:15.282831 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:15.282801    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-msjdg_70f17e22-92d6-46f7-b95f-1ffc1fd2731b/insights-operator/0.log"
Jul 24 16:42:15.283502 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:15.283485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-msjdg_70f17e22-92d6-46f7-b95f-1ffc1fd2731b/insights-operator/1.log"
Jul 24 16:42:15.438844 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:15.438822    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pct5v_11be8611-517c-405e-a8a3-4012c59d082a/kube-rbac-proxy/0.log"
Jul 24 16:42:15.458323 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:15.458301    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pct5v_11be8611-517c-405e-a8a3-4012c59d082a/exporter/0.log"
Jul 24 16:42:15.477568 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:15.477549    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pct5v_11be8611-517c-405e-a8a3-4012c59d082a/extractor/0.log"
Jul 24 16:42:16.783015 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:16.782985    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6ww8s/perf-node-gather-daemonset-4z5jb"
Jul 24 16:42:18.098628 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:18.098592    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749946f7d7-c4ps8_08737d3f-3b02-4c1e-a82d-c70113b2f206/manager/0.log"
Jul 24 16:42:18.119966 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:18.119941    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749946f7d7-c4ps8_08737d3f-3b02-4c1e-a82d-c70113b2f206/copy-manifests/0.log"
Jul 24 16:42:19.718861 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:19.718785    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-777c588886-w7ntt_b0c5880a-e163-4101-a031-8f2f302e1dc8/manager/0.log"
Jul 24 16:42:19.748122 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:19.748101    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-j9v6g_47357e38-5ac3-4ea2-b7ba-039a343bde53/openshift-lws-operator/0.log"
Jul 24 16:42:20.291383 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:20.291357    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-f8fhw_f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e/discovery/0.log"
Jul 24 16:42:20.440020 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:20.439995    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-747d9dcfc7-cs2nw_e1a4672e-a764-4839-82de-f0a45a3ed786/router/0.log"
Jul 24 16:42:21.971542 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:21.971477    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-f8fhw_f1bb2616-d2d9-49d7-bab8-3ac80f3cea1e/discovery/0.log"
Jul 24 16:42:22.110867 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:22.110842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-747d9dcfc7-cs2nw_e1a4672e-a764-4839-82de-f0a45a3ed786/router/0.log"
Jul 24 16:42:26.150180 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.150154    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/kube-multus-additional-cni-plugins/0.log"
Jul 24 16:42:26.175854 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.175829    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/egress-router-binary-copy/0.log"
Jul 24 16:42:26.195501 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.195474    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/cni-plugins/0.log"
Jul 24 16:42:26.214601 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.214583    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/bond-cni-plugin/0.log"
Jul 24 16:42:26.233586 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.233566    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/routeoverride-cni/0.log"
Jul 24 16:42:26.253146 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.253128    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/whereabouts-cni-bincopy/0.log"
Jul 24 16:42:26.272194 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.272172    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njndp_97df7b64-4b55-47b8-b30c-956abe38b044/whereabouts-cni/0.log"
Jul 24 16:42:26.488931 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.488878    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sp4d8_20450065-d354-4132-99f9-4f5b0fdc7196/kube-multus/0.log"
Jul 24 16:42:26.554168 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.554151    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pc2tq_a854e3ef-7cf7-467e-896b-339ed5f3acc6/network-metrics-daemon/0.log"
Jul 24 16:42:26.571519 ip-10-0-133-118 kubenswrapper[2577]: I0724 16:42:26.571505    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pc2tq_a854e3ef-7cf7-467e-896b-339ed5f3acc6/kube-rbac-proxy/0.log"